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ABSTRACT 
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D'i. 

NA' 


Questions regarding the usefulness of the Naval Postgraduate 
School's Student Opinion Form (SOF) as a device to measure 
teaching effectiveness have prompted this research. The 
possibility that the SOF may weigh heavily in pay, promotion, 
and tenure decisions is cause for research into its validity 
and reliability as an evaluation instrument. The first of 
three separate studies described here consists of an analysis 
of a questionnaire distributed to all teaching professors in 
the NPS Administrative Sciences Department. The second study 
concerns a questionnaire completed by 258 Administrative 
Sciences students, and the third study considers the responses 
of 560 students to four supplementary items added to the SOF. 

The results indicate that neither students nor faculty 
members feel strongly that SOF's actually measure or improve 
teaching effectiveness, that a large part of the variation 
in SOF ratings is attributable to factors other than a pro¬ 
fessor's teaching quality and, finally, that a student's 
anticipated course grade or cumulative grade point average 
has little correlation with the SOF ratings given the 
professor. 
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I. INTRODUCTION 


A. MOTIVATION 

In recent years, a great deal of interest has been 
expressed about evaluating the quality of instruction. 

Reasons for this interest appear to include decreasing college 
enrollments, increasingly demanding educational goals, and 
a growing emphasis on instructional accountability. Since 
1975, the Student Opinion Form (SOF) has been used at the 
Naval Postgraduate School as a tool to measure the teaching 
effectiveness of instructors. Student-faculty evaluations 
(SFE’s) of instructor effectiveness have become widely 
accepted in the academic world as helpful indicators of 
performance, but there has been considerable controversy 
regarding their validity for use in pay, promotion, and 
tenure decisions. In addition to the questionable validity 
of SFE’s, another common criticism is that they are biased 
by variables that are unrelated to teaching effectiveness. 
Frequently, however, computer formatted students' evaluations 
are the only form of faculty evaluation used, mainly because 
of ease of administration and processing. 

Many college faculty fear that too much emphasis on 
student-faculty evaluations may lead to manipulative and 
other corruptive practices that would have a negative impact 
on the quality of education and on the academic community 
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in general. These concerns are certainly not new to the 
academic world. In fact, these problems have plagued 
educators as far back as medieval times. Historical writings 
about medieval European universities discuss the restrictive 
methods resorted to in an attempt to prevent these poten¬ 
tially corruptive practices. Professors were paid a "col- 
lecta** or fee by each student to teach an agreed-upon amount 
of material by a specified date. A group of students was 
appointed by the rector to report negligent professors who 
had not fulfilled their contractual agreement. Based upon 
this group's evaluation, a fine would be imposed upon the 
professors for each day they had fallen behind in their 
teachings. Although this may be viewed as an extreme response 
to the problem, many educators today question the ultimate 
impact of student evaluations on teaching effectiveness. 

These concerns, as well as the potential benefits to be 
gained from student-faculty evaluations, have triggered 
numerous studies. 

B. RESEARCH 

Research interest in student-faculty evaluation (SEE) 
has increased significantly in recent years. This concen¬ 
trated attention not only is due to student interest but 
also seems to be a function of the increased use of SEE 
in the determination of faculty promotions and salaries. 

In fact, the agencies that govern public colleges and 
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universities in some states require that SFE be employed 
as one component in pay, promotion, and tenure decisions. 

This requirement is apparently based on the premise that 
SFE provides a precise and quantifiable evaluation of in¬ 
structors. However, the assignment of numbers to instructor/ 
course attributes does not automatically confer reliability 
or validity upon the rating system as a whole. There have 
been numerous studies with extremely inconsistent results 
as to the relationship between SFE and teaching effective¬ 
ness (as measured by amount learned). [Ref. 1] Studies that 
resulted in both highly positive [Ref. 2] and highly negative 
correlations [Ref. 3] have been reported; however, the majority 
of these studies yielded correlations that did not differ 
significantly from zero [Ref. 4]. 

The major aim of student-rating proponents is basically 
to obtain more valid, reliable, and effective means of 
incorporating the evaluation of teaching into advancement pro¬ 
cedures that might otherwise be available. These proponents 
also feel that there are many benefits to be gained from 
these evaluations. Specifically, Costin, Greenough, and 
Menges, in an extensive review of reliability, validity, and 
usefulness of student ratings, state that: 

1. Such ratings could provide feedback which the in¬ 
structor might not be able to elicit from students on 
a face-to-face basis. (This information alone, with 
no sanctions contingent, could improve teaching.) 
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2. They could provide departmental and college-wide 
norms against which individual faculty ratings could 
be judged, 

3. They could provide a way in which a faculty member 
could, if he desired, demonstrate his undergraduate 
teaching effectiveness to those who have expressed an 
interest in evaluating this parameter for salary 
increase. 

4. They could provide information to the department and 
college on areas of relative strength or weakness in 
undergraduate teaching, suggest directions for the 
development of new courses or programs, and provide 
evaluative information and norms on the various new 
programs which are implemented. 

5. They could provide the student with a source of 
information to aid him in the selection of courses. 

[Ref. 5] 

It should be noted that these benefits can exist only to the 
extent that student-faculty evaluations portray accurate and 
valid appraisals of classroom instruction. 

Mas low and Zimmerman, when asking students to make 
global ratings of teachers' effectiveness, provided the fol¬ 
lowing definitions to aid students in making their ratings: 

. . . A person deserving the highest possible rating, 
as a teacher, was described as one who is both capable 
and efficient, who loves his job and manages to inspire 
his students, who is himself inspired with his work, 
who is talented, and who not only respects and appre¬ 
ciates his students, but also has good relations with 
them. The highest rating as a personality was to be 
given to a very healthy, well integrated person, sub¬ 
jectively at ease with himself, happy or content, using 
all his constructive capacities, enjoying life without 
neurotic or psychotic maladjustments. [Ref. 6] 

Though Maslow and Zimmerman found a 0.76 correlation between 

students’ ratings of "good teaching" and "good personality," 

both personality and ability were so vaguely defined as to 
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cause difficulty in interpreting the results with any 
precision. [Ref. 7] 

It cannot be denied that SFE is influenced by many non¬ 
ins tructional variables, which, more than likely, account for 
the inconsistent findings, as noted above. Variables such 
as class size, course level, presentational style, actual or 
expected grade, and student group are all likely major deter¬ 
minants of SFE. For instance, an interesting example of the 
effect of presentational style and format was demonstrated 
at one school in which a professional actor was hired to 
deliver a series of lectures to a large student group. 

Because of his enthusiastic, humorous, and personable manner, 
he received extremely high ratings; though entertaining, 
educational content of his presentation was low or totally 
absent. [Ref. 8] The favorable rating in this case can be 
considered to be a function of superficial popularity or 
what is sometimes referred to as the "popularity halo." 

In another study. Brown (1976) found that student grades 
accounted for approximately nine percent of the variance 
among student ratings (on the average across classes). It 
was his interpretation that, as a result of this small but 
positive relationship between student grades and student 
ratings, professors can technically "buy" high student ratings 
by giving their students high grades. [Ref. 9] This is a 
very popular interpretation of the frequent low correlations 
and poor regression results that are obtained. Conversely, 
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Voeks and French have found in their studies that "high ratings 
cannot be 'bought' by giving high grades, nor are they lost 
by giving low grades." [Ref. 10] This finding would imply 
that college students have greater objectivity and less 
superficial value systems than they have been given credit 
for by others. 

Howard and Maxwell (1980) have proposed three models 
that attempt to bridge the gap between these two schools of 
thought. [Ref. 11] Their first model (Figure 1) very simply 
states that students' evaluations are directly dependent on 
their expected or actual grades. 



Figure 1. Grading Leniency Bias Model 

Their second model (Figure 2) suggests that greater teaching 
effectiveness causes improved student performance, which 
consequently results in both higher student grades and 
student evaluations. Howard and Maxwell's final model 
(Figure 3) proposes that greater student motivation causes 
better student performance, resulting in higher student grades 
and higher student evaluations. 
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Figure 2. Teaching Effectiveness Model 



Figure 3. Student Characteristics Model 
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Figures 2 and 3 both exhibit a positive relationship 
between student grades and student evaluations. However, 
this association is interpreted as a logical result o£ student 
performance and not bias. As can be seen, there is little 
agreement among researchers about the effects of student 
grades on student evaluations, and even concerning the inter¬ 
pretation of the research results. 

Several researchers have also focused their studies in the 
direction of instructor personality traits. Results in this 
field reveal that "overall effectiveness" of teaching, as 
perceived by students, seems to be positively related to 
instructor characteristics (e.g., imagination, intelligence, 
emotional stability, enthusiasm). For example, Hildebrand 
and Wilson conducted a highly acclaimed research project of 
this kind. Their main goal was to develop a reliable instru¬ 
ment that would provide a basis for evaluating teaching that 
could also be incorporated into advancement procedures. 

Their report discusses the development of three forms of 
varying lengths that could be used as teaching evaluation 
instruments for the California State University system. Both 
student and faculty characterizations of effective teaching 
were assessed, but the final recommended instrument relied 
heavily on the students’ characterization. Five scales 
(teacher description scales), established from this instru¬ 
ment via factor analysis, have the following conceptual 
interpretations: 
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1. Analytic/Synthetic Approach--scholarship, with 
emphasis on breadth, analytic ability, and conceptual 
understanding. 

2. Organization/Clarity--skill at presentation, but is 
subject-related, not student-related, and not merely 
concerned with rhetorical skills. 

3. Instructor-Individual Student Interaction--rapport 
with the class as a whole, sensitivity to class response, 
and skill at securing active class participation. 

4. Instructor-Individual Student Interaction--mutual 
respect and rapport between the instructor and the 
individual student. 

5. Dynamism/Enthusiasm--flair and infectious enthusiasm 
that comes with confidence, excitement for the subject, 
and pleasure in teaching. [Ref. 12] 

These five scales have become widely acknowledged and several 

researchers have based their studies on them. 

Standard characteristics currently found on over twenty- 
eight student-faculty evaluations were reviewed in this 
study. The following factors seemed to be the most relevant 
and popular: course/lecture organization, instructor atti¬ 
tude toward student understanding, instruction preparation, 
clarity of explanation, instructor knowledge, instructor 
evaluation of student, ability to teach at appropriate level, 
instructor attitude/enthusiasm toward course, instructor 
attitude toward questions, instructor control of class time, 
instructor availability, instructor ability to evoke interest, 
clarity of course objectives, instructor attitude toward 
student progress, rapport with students, course workload, 
and, finally, relations of subject matter to real world 
applications. There also have been studies that numerically 
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rank, by importance, the above factors by polling student 
groups. The majority of these studies found course organi¬ 
zation to-be the most important factor. 

Other studies have taken a slightly different approach 
and have attempted to determine a set of mutually exclusive 
factors that are relevant to satisfaction or dissatisfaction 
with teacher performance. For example, Elster ejt used 
the critical incident technique to determine the variables 
that lead to students' satisfaction with instructors at the 
Naval Postgraduate School. The students (over 250) were asked 
to identify a time when they were especially satisfied or 
dissatisfied with a teacher and to describe what had led 
them to feel that way. A set of collectively exhaustive 
and mutually exclusive content categories were developed. 

The category labeled "organization and preparation" was the 
most frequently appearing response, followed by "instructor's 
traits" and "instructor's classroom or presentation tech¬ 
niques." The "instructor's knowledge of the subject" and 
"evaluation of students in general" were also categories 
that were frequently mentioned. [Ref. 13]. 

Similarly, Gadzella (1968) asked a randomly selected 
sample of students to state their opinions of an "ideal 
professor." This study found that the four most important 
criteria were knowledge of subject, interest in subject, 
flexibility, and preparation. [Ref. 14]. 
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Costin (1968) asked over 200 graduate and undergraduate 
students from three different universities to rate statements 
regarding the classroom behavior of the "best lecturer" 
that they had ever encountered. A four-point integer scale 
ranging from (4) "almost always occurred" to (1) "almost 
never occurred" was used. Items such as "acted interested 
in the material," "was well prepared," "used relevant examples," 
"followed a logical sequence of thought," and "explained 
clearly" received mean score ratings above 3.5. [Ref. 15] 

It has been disputed that students' judgments of 
desirable criteria of teaching effectiveness many times are 
immature and inaccurate and lack long-term perspective. This 
point has been discredited on numerous occasions. For example, 
Drucker and Remmers asked college alumni what they thought 
the most important qualities of a good instructor were, and 
compared the answers with current undergraduates' views. 

They found that both groups agreed identically on the 
ranking of these criteria: 

a. adequacy of preparation 

b. interest 

c. stimulation of intellectual curiosity 

d. "progressive" attitude 

In addition, they found that student ratings correlated 
positively with alumni ratings of the same instructors 
[Ref. 16]. 
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In summary, much research has been completed in the above 
areas of student-faculty evaluation. Although researchers' 
approaches may differ, they are all, in essence, attempting 
to determine the most reliable, valid, and appropriate 
method of student-faculty evaluation. 

C. SOF HISTORY AND DEVELOPMENT 

The Student Opinion Form (SOF), which is presently used 
at the Naval Postgraduate School, was developed locally and 
instituted in June of 1975, This particular form replaced 
the Student Instruction Report (SIR), which was a widely 
used form obtained from the Educational Testing Service (ETS) 
at Princeton. The SIR elicited students' judgments of such 
aspects as the organization of the course, the pace of the 
course, the instructor's helpfulness or availability to 
students, and the clarity of objectives and presentations. 

The SIR contained 39 questions while the presently-used SOF 
contains 16 questions as well as a provision for written 
comments. The free-form comments afford the student a means 
of additional communication and an opportunity to convey 
specific feedback to the instructor (which is available only 
to the instructor). The SOF was adapted from the technique 
of Hildebrand and Wilson (1971) and formatted after SIR, 

More specifically, seven of the questions on the current SOF 
were taken directly from the SIR form and the remaining ques¬ 
tions were added by an appointed committee of NPS professors. 
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The present SOF system requires that these SOF forms 
be distributed at the end of each term and that all students 
individually fill out the sixteen-item questionnaire for 
each course that they are enrolled in. Items one through 
eleven are behavioral in nature while items twelve through 
sixteen are •'overall" ratings. These questionnaire items 
are: (1) The course was well organized; (2) Time in class 

was spent effectively; (3) The instructor seemed to know when 
the students did not understand the material; (4) Difficult 
concepts were made understandable; (5) I had confidence in 
the instructor's knowledge of the subject; (6) I felt free 
to ask questions; (7) The instructor was prepared for class; 
(8) The instructor's objectives for the course have been 
made clear; (9) The instructor made this course a worthwhile 
learning experience; (10) The instructor stimulated my 
interest in the subject area; (11) The instructor cared 
about student progress and did his share in helping us learn; 

(12) Overall rating of the instructor; (13) Overall rating 

of the course; (14) Overall rating of the textbook(s); 

(15) Overall rating of the quality of the exams; and 

(16) Overall rating of the laboratories. All of these 

items are scored on an integer scale from zero to five. The 
response options for the behavioral items are: no comment, 
strongly disagree, disagree, no strong opinion, agree, and 
strongly agree, respectively; the response scale for the 
overall items is as follows: N/A, poor, fair, average. 
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excellent, and outstanding, respectively. Appendix A shows 
an actual SOF used in this study for analysis purposes. 

The NFS SOF's were processed in-house for several years 
via an optical scanning machine. Numerous mechanical break¬ 
downs of the op-scanner, as well as machine processing 
inaccuracy, resulted in a great need for a more flexible and 
accurate system. In the spring of 1983, processing was 
delegated to a civilian contractor, McGraw-Hill, which 
enabled the Naval Postgraduate School to receive results 
approximately one week following submission. Processing 
costs of the SOF run approximately $.25 per page. NPS sub¬ 
mits between 6000 and 7000 forms per quarter for optical 
scanning at McGrawHill. 

Student-faculty evaluations at NPS were initially 
intended to be used strictly as feedback for the individual 
professors to aid in the improvement of instruction. They 
have since evolved into a supportive evaluation tool to be 
used by faculty administrators as an overall aid in pay, 
promotion, and tenure decisions. This evolution into an 
administrative use has created considerable controversy. 

Zelby states that one major cause for the growth of the 
student-faculty evaluation controversy is the "trend toward 
formal, quantitative use of the results of the evaluations 
in determinations of faculty promotions and salaries." He 
states further that there are three reasons for this trend: 
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(1) SFE provides documented, precise, empirical evalua¬ 
tions of instructors; (2) it tends, thereby, to relieve 
academic administrators from the responsibility of 
exercising judgment about teaching performance and 
ability; and (3) it tends to constitute proof that 
something, indeed, is being done to improve teaching. 

[Ref. 17] 

In 1977 and 1978 there seemed to be a growing interest 
on the part of NFS faculty and students in the Student 
Opinion Form. In April of 1977, the NFS Scholarship 
Committee submitted a memorandum to the Faculty Council 
recommending, among other things, the initiation of a statis¬ 
tical research project to determine the significance of SOF 
scores. In addition, the Student Council took the initiative 
to prepare a SOF information sheet for dissemination to the 
student body based on information acquired from NFS faculty 
administrators. Appendix B contains a copy of the Student 
Council's memorandum. Interest was so great as to evoke 
the development of a proposed new Student Opinion Form. This 
alternative form was never instituted at NFS, though it 
offered many innovative ideas. 

When the SOF was initially developed, strict guidelines 
were outlined for its administration. Unfortunately, in the 
past ten years there has been a decline in the emphasis 
placed on the careful adherence to these guidelines, result¬ 
ing in the present lack of consistency in SOF administration 
among course segments. 

The initial guidelines required the SOF's to be com¬ 
pleted within the last two weeks of the quarter, prior to 
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finals, and during class time. One SOF was to be filled out, 
in the absence of the professor, by each student for each 
class segment he or she took for course credit. Specific 
instructions for completion of the SOF were provided on 
each form (e.g., soft black pencil, no ink). The SOF forms 
were to be disseminated by the professor and collected by 
the appointed section leader at the end of the allotted 
time. Once collected, the SOF's were to be placed in an 
envelope and immediately submitted to the department office. 
Neither the professor nor the section leader was to review 
the completed forms. Once the processing of the SOF was 
completed, summary statistical data were provided to the 
department chairman and the actual SOF's were returned to 
the professor. 

The original guidelines are still in effect; however, 
because of inadequate guidance and lax enforcement, little 
consistency in SOF administration exists in the system today. 
It is not unusual for the SOF's to be filled out at home, 
completed at a later date (sometimes well into the following 
quarter), or never completed at all. This problem has been 
compounded by the frequent complaints of insufficient time 
being provided to fill out the SOF's. There are also 
numerous instances of the professor remaining in the class¬ 
room and wandering amongst the students during SOF completion. 
Another problem is that students tend not to read the 
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instructions and, hence, to fill out the computer forms 
incorrectly. In addition, extraneous markings or comments 
are frequently made in areas other than those designated. 

It also has become commonplace for the section leaders 
and other student members to review the completed SOF's 
in their entirety prior to submission. Because of the lack 
of specific and comprehensive instructions or other guidance, 
many of the above infractions contribute to further poor 
compliance, as can be seen in the mean SOF return rate. As 
an example, the SOF percentage return rate in the Administrative 
Sciences Department for the past eight years has averaged 761. 
(See Appendix C.) Despite a slightly increasing return rate 
trend over this period, there is still considerable apathy 
among students toward SOF's. Unless greater emphasis is 
placed on the careful completion of SOF forms and new 
interest rekindled in their validity as an evaluation tool, 
their usefulness if not their use will continue to decline 
until they serve no further purpose. 

In addition to these problems, there are several other 
concerns with the processing and utilization of the SOF. 

As was previously stated, processing time at McGraw-Hill re¬ 
quires approximately one week; therefore, it was anticipated 
that the completed SOF data would be returned to the faculty 
during the first week of the following term. Unfortunately, 
a one-and-a-half to two-month turnaround time has become 
reality. This delay is primarily due to careless completion 
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of SOF*s, thus requiring extensive editing. At present, 
limited available resources allow for only one editor to 
make these corrections. Since nearly 201 of the 6000 com¬ 
pleted SOF's require editing, this job generally exceeds 
one month. In essence, it was unrealistic to expect a one- 
to two-week turnaround time; however, this tremendous edit¬ 
ing problem was never expected. 

By the time the evaluations are available to the pro¬ 
fessors, the school is well into the following term. The 
SOF information by this time may be virtually irrelevant, 
since the instructors are most likely teaching a different 
student group and possibly a different course. In any case, 
feedback received at the beginning of the quarter would 
allow sufficient time to make changes in course structure, 
exams, or even possibly teaching approach. 

D. DIRECTION OF RESEARCH 

The review of much of the previous research in the area 
of SFE, as well as the history of NFS's own SOF, led to an 
inquiry into several areas related to SFE. Over the past 
nine years, it appears that interest in SFE has waned. 

During the time between 1965 and 1975, it was evident that 
there was a burgeoning concern regarding the quality of 
college and university teaching. Prior to this time, research 
and publication were considered to be of primary importance in 
the evaluation of a professor's capabilities. 


26 




Historically, every generation of students has been hungry 
for good teachers, but the current generation has been the 
first in this country to mount an organized attack on what 
they charge has been the Establishment’s lack of concern 
for the poor quality of college instruction. [Ref. 18] 

In order for quality of teaching to improve, a reliable method 

for measuring teaching effectiveness must be incorporated 

into the educational system. 

The situation just described prompted the authors of this 
paper to address these issues pertaining to SEE at the Naval 
Postgraduate School. Three individual studies were pursued 
in this area. All data collected were restricted to the NPS 
Administrative Sciences Department. The first study con¬ 
cerned a professor questionnaire on the NPS Student Opinion 
Form that was distributed to fifty instructors who were 
then teaching at least one course. The second involved a 
student questionnaire on the SOF that was disseminated 
to 258 students. Finally, in the third study, four addi¬ 
tional questions were added to the currently-used NPS SOF 
for the purpose of gathering further SOF information. 


27 



























II. PROFESSOR QUESTIONNAIRE 


A. MOTIVATION 

The decision to design and administer a professor ques¬ 
tionnaire was primarily motivated by candid comments that 
emerged from informal conversations with professors from 
different departments at the Naval Postgraduate School. In 
addition, myths and rumors regarding the use and purpose of 
Student Opinion Forms have abounded among students and 
faculty. There is as much variation in opinion by the pro¬ 
fessors as there is among students on the value of the SOF 
as a feedback and evaluation tool. Hutchison cautions 
against the use of student-faculty evaluations for evalu¬ 
ation purposes because the students are, in essence, merely 
reporting perceptions, not making performance appraisals. 

[Ref. 19] It was our general impression that many professors 
seriously question the validity and reliability of the SOF, 
and place little credence in the SOF results, 

A study by Herbert W. Marsh indicates that this skepti¬ 
cism about student-faculty evaluations is not uncommon among 
faculty members. 

Faculty are concerned about teaching effectiveness, 
even to the extent of wanting it to play a major role 
in administrative decisions, but have no confidence 
in any measures of teaching effectiveness--including 
students' evaluations .... An important role of 
research in students' evaluations, besides demon¬ 
strating their reliability, validity, and lack of 
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bias, is to convince faculty and administrators of 
their worth. [Ref. 20] 

For example, some professors feel the senior class member (or 
section leader) has a great deal of influence over the ratings 
received by the professor. It has been rumored that in some 
cases the class section leader has organized meetings of the 
class members to arrive at a common "grade" for the professor's 
SOF. (This may be true, especially if the professor has not 
lived up the expectations of the class or, in particular, the 
class leader.) Such collusive practices only serve to support 
faculty reservations about the accuracy of the students' 
evaluations. 

Another study, by Robert R. Read of the Naval Postgraduate 
School, revealed that the effects of two factors other than 
the professor's actual teaching ability, student group and 
the particular course, enter into the SFE ratings. Read 
found through analysis of variance that, of these three 
factors, the effect of the course was of supreme importance; 
and, surprisingly enough, the effect of the student group 
also surpassed the effect of the professor. Professor Read 
concluded that Item 12 of the NPS SOF (overall rating of 
instructor) does not measure what it claims to measure because 
of the strong course and student-group components of variation 
in response to this particular item. [Ref. 21] 

In spite of this lack of confidence in the validity of 
student-faculty evaluations, they are often the only 


29 



f 


available measure of teaching effectiveness; and professors 
and administrators are therefore led to rely upon them. 

This reliance is evidenced by the frequent use of student- 
faculty evaluations for pay, promotion, and tenure purposes. 
Wilson, Gaff, and Bavry (1970) surveyed one thousand faculty 
members from six schools regarding the criteria that are used 
by college administrators in making advancement decisions. 
Ninety-two percent of those surveyed felt that teaching 
effectiveness should be quite important or very important in 
promotion decisions, although only 38% stated that teaching 
effectiveness actually is considered important to this extent 
in advancement decisions. Seventy-two percent of the faculty 
members surveyed felt that a formal evaluation procedure 
should be instituted at their schools. [Ref. 22] Of course, 
research and publications play an important role in these 
decisions; however, there are no formal quantitative methods 
of evaluation used at NFS to measure these particular com¬ 
ponents of professor performance. 

Although the SOF covers a wide variety of items, there 
is a noted tendency to concentrate on SOF Item 12, the over¬ 
all rating of the instructor. In particular. Item 12 is 
relied upon by some department chairmen as a "quick and dirty" 
overview of the professors* teaching capabilities. Not unlike 
notification of students who make the Dean's List, congratula¬ 
tory letters are sent to NFS faculty whose mean score on this 
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particular item is high. In fact, at least one department 
at the Naval Postgraduate School rank-orders its professors 
based solely on their mean Item 12 score. This practice may 
be a self-defeating one because of the potential detriment 
to the morale and self-esteem of professors who rank low. 

Such great emphasis on the Student Opinion Forms has 
prompted professors to resort to manipulative practices to 
obtain higher SOF scores from students. In one specific 
example, a professor deliberately failed to teach a particu¬ 
larly difficult portion of a course in an effort to enhance 
his SOF scores. It was his announced perception that higher 
SOF scores might lead to a step (merit) increase in salary. 
Rodin and Rodin's study found that students rate most highly 
those professors from whom they learned the least, thus 
supporting this professor's premise. [Ref. 23] 

It was found that the professors of the NPS departments 
that typically had the highest mean scores for SOF Item 12 
made a specific point of informing their students that the 
SOF's were used for pay, promotion, and tenure decisions. 
Identical results were obtained in a study at the University 
of Wisconsin, Green Bay, where researchers found "SFE ratings 
increased in all evaluation factors when students were in¬ 
formed that the SFE is used for salary, promotion, and 
retention . . . ." [Ref, 24] 
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B. APPROACH 


The professor questionnaire was distributed to all 
Administrative Sciences professors who were teaching classes 
of five or more students, one questionnaire per class section. 
Fifty questionnaires were disseminated, and 38 completed 
questionnaires were returned, a return rate of 76%. The 
questionnaire consisted of 23 questions, 21 multiple choice 
and two fill-in-the-blank. 

It was the authors' intention to poll the professors on 
their beliefs and opinions regarding the Student Opinion 
Forms. The questions ranged from general ones about the 
course, the students in the class, and the class section leader 
to personal questions pertaining to academic rank and tenure. 
There were also questions regarding the purpose and useful¬ 
ness of the SOF and the amount of weight SOF's should carry 
in pay, promotion, and tenure decisions. Appendix D contains 
a copy of the actual questionnaire and the frequencies with 
which the professors selected each item option. The follow¬ 
ing is a synopsis of the professors' responses to the 
questionnaire : 

* Of the professors who completed the questionnaires, 

21% were teaching Financial Management courses, 16% 
each were teaching Manpower/Personnel Management, 
Organizational Effectiveness, and Economics/Account¬ 
ing courses, 11% Information Systems, and 10% other 
courses. 

* The majority of the students in the classes whose 
professors were surveyed were in the Manpower/Personnel/ 
Training Analysis, Financial Management, or Computer 
Systems curricula. 
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* Seventy-six percent of the professors queried were 
not aware of who the class section leader was. 

* Of those who were cognizant of the section leader's 
identities, nearly 60% anticipated those student leaders 
to fall in the top half of the class, gradewise. Most 
of the professors did not feel that the section leaders 
had much influence over the other students in the class. 

* Nearly all professors (85%) thought the students 
liked their courses at least fairly well, and 95% 
enjoyed teaching their courses. 

* Over half of the professors surveyed had been at the 
Naval Postgraduate School less than three years, and 
most (76%) did not have tenure. 

* Seventy-nine percent of the professors were teaching 
courses that they considered to be in their area of 
professional expertise. 

* Well over half of the respondents believed that SOF 
forms currently carried no more than 40% of the weight 
in pay, promotion, and tenure decisions; 13% had no 
idea. 

* Seventy-four percent of the faculty believed that the 
SOF should carry no more than 40% of the weight in pay, 
promotion, and tenure decisions. 

* The preponderance of professors did not feel that 
SOF's actually measure teaching effectiveness to any 
great extent, and they only found SOF's somewhat 
useful in improving their own teaching effectiveness; 
in fact, only 14% of the faculty believed that the SOF 
measures teaching effectiveness to a large or a very 
large extent. 


C. METHODOLOGY 

In addition to general interest in the professors' 
knowledge and opinions about the SOF, the authors also 
hoped to predict each professor's mean SOF Item 12 score, 
by course segment, based on his/her questionnaire responses. 






















It was the authors' hypothesis that a combination of selected 
factors from the questionnaire would have the capability of 
explaining a large proportion of the variation in SOF Item 12 
scores . 

Aggregated data were obtained from the Student Opinion 
Forms completed by the students in the classes for which the 
professors returned questionnaires. Hence, the authors were 
able to join the required data from both sources to perform 
this statistical analysis. 

The Statistical Package for the Social Sciences (SSPS) , a 
system of computer programs designed specifically for use in 
analyzing data from the social sciences, was utilized in the 
study. SPSS allowed for the treatment of missing values, which 
was of great benefit in this study because of the small initial 
sample size and the considerable number of missing values. Two 
of the questions in the survey represented categorical rather 
than quantitative variables, so the SPSS Breakdown program was 
used to replace the qualitative responses by their Item 12 
means to provide a usable, numerical format. The Breakdown 
program allows the user to obtain means, standard deviations, 
and variances of a numeric dependent variable for each category 
of an independent variable. In the case of Question 1 of the 
professor questionnaire, "What is the general subject area of 
this course?". Breakdown provided the mean SOF Item 12 score 
for all professors who selected each offered response. For 
example, the mean SOF Item 12 score for all professors who 
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responded that they were teaching Manpower/Personnel Manage¬ 
ment courses was 4.13 on an integer scale of 0 to 5 (0 
being ”N/A'’, 5 being ''outstanding'*) . 

Once the questions were all in a quantitative format, 

15 of the 23 were selected as independent (predictor) 
variables for use in the statistical analysis. SOF Item 12 
was the dependent (criterion) variable. In the first re¬ 
gression, the 15 independent variables selected consisted of 
both "structural" and "subjective" questions. The "struc¬ 
tural" questions were identified as those over which the 
professor had little or no control, such as class size, 
number of courses he/she was teaching that quarter, or 
general subject area of the course. The "subjective" 
questions were defined as those in which the professors had 
control, such as knowing who the class section leader/senior 
officer was, or those questions that requested a judgment 
or viewpoint, such as whether the professors felt they had 
good rapport with the students in the class. The second 
regression involved only ten "structural" independent 
variables, in addition to SOF Item 12, again, as the 
dependent variable. 

Table I lists the questions from the professor question¬ 
naire. The "structural" questions are followed by (ST), 
and the "subjective" questions are followed by (SUB). The 
variables that were included in the regression equations are 
also followed by an asterisk (*). 
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Table I 


Professor Questionnaire Items 


(Ql) What is the general subject area of this course? 
(ST)* 

(Q2) What is the academic level of this course? (ST)* 

(Q3) How many students are enrolled in this class? 

(ST)* 

(Q4) What is the curriculum number of the majority of 
this class? (ST)* 

(Q5) Do you know who the section leader or senior officer 
in this class is? (SUB)* 

(Q6) To what extent do you feel the section leader or 
senior officer has influence over the class in 
relation to the course or the instructor? (SUB) 

(Q7) Gradewise, what quartile do you expect the section 
leader or senior officer to fall in? (SUB) 

(Q8) How well do you think the students generally like 
this course? (SUB)* 

(Q9) How many years have you been on the staff at the 
Naval Postgraduate School? (ST)* 

(QIO) What is your academic rank? (ST) 

(Qll) Do you have tenure? (ST)* 

(Q12) What step are you presently on the faculty salary 
schedule? (ST) 

(Q13) To what extent do you consider this course to be in 
your area of professional expertise? (ST)* 

(Q14) How many courses (exclusive of labs) are you 
teaching this quarter? (ST)* 

(Q15) How many times have you taught this course prior 
to this quarter? (ST)* 
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(Q16) Have you ever taught this student group (plus or 
minus a few students) before this quarter? (ST)* 

(Q17) How well do you like teaching this course? (SUB)* 

(Q18) To what extent do you feel that you have good 
rapport with this class? (SUB)* 

(Q19) Do you tend to organize and present this course 
differently according to the seniority of the 
student group that you are teaching (e.g., first 
quarter vs. last quarter thesis students)? (SUB) 

(Q20) How much weight (percentage-wise) do you think the 
SOF's presently carry in pay, promotion, and tenure 
decisions? (SUB) 

(Q21) How much weight (percentage-wise) do you think the 
SOF's should carry in pay, promotion, and tenure 
decisions? (SUB) 

(Q22) To what extent do you feel SOF's actually measure 
teaching effectiveness? (SUB)* 

(Q23) How useful do you feel the SOF is in improving your 
teaching effectiveness? (SUB) 


Because of the small sample of 38 completed question¬ 
naires that were available for statistical analysis, a step¬ 
wise regression was decided upon. This procedure allows 
a researcher to isolate a small subset of the available 
independent variables that will result in an optimal predic¬ 
tion equation. The SPSS Stepwise Regression program allows 
the researcher, in particular, to specify three statistical 
criteria to be used in determining which predictor variables 
will be selected. The first parameter stipulates the 
maximum number of predictor variables that will be included 
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in the equation. The second parameter identifies the minimum 
F value (which is automatically computed to test for the 
significance of the regression coefficient) that the researcher 
will accept for variables that are to be entered in the 
regression. The third stipulation specifies the tolerance, 
or one minus the squared multiple correlation that a candi¬ 
date independent variable has with the other independent 
variables already selected. It was decided that a total of 
five variables would be selected, with the default values 
for both the F-test (.01) and the tolerance (.001) used so 
as to minimize the restrictions placed on the stepwise 
regression. 

Missing data for the Breakdown analysis was handled by 
eliminating only the cases in which the dependent variable 
was missing. (In all other instances, the case was included.) 
In the regression analysis, the default option, listwise 
deletion of missing data, was used. This option auto¬ 
matically deletes all cases in which any missing values 
occur. Thus, all statistics computed in conjunction with the 
regression program were based only on the remaining cases. 
Because of the numerous missing values encountered, the small 
sample size used in this study was narrowed down to only 
28 cases through listwise deletion. 

As part of the regression program, a table of correla¬ 
tion coefficients was provided to check for multicollinearity 
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among the independent variables. It is important to minimize 
the effects of multicollinearity so as not to confound the 
relative importance of independent variables entered in the 
regression equation. With only a few exceptions, there 
were very low intercorrelations between the predictor 
variables, the majority falling below 0.35. 

The first regression analysis (involving both ’’structural'* 
and "subjective" variables) entered Question 4 on step one. 

The F value, indicative of statistical significance, was 
high, at 18.83, with a very impressive multiple R of 0.65. 
Thus, with just one variable entered, over 42% of the varia¬ 
tion in mean SOF Item 12 scores was explained. Step two 
entered Question 8, increasing the multiple R to 0.76. 

Question 1 was included in step three, with a multiple R of 
0.79, and step four brought in Question 17, to raise the 
multiple R to 0.82. Although Question 16 was entered in 
step five, the F value for this entry was only 0.82. 

Because of the low statistical significance of this value, 
only the first four variables were used in constructing the 
regression equation. 

In the second regression analysis (involving only the 
"structural" variables). Question 4 was, again, the first 
variable to be entered, with an F value of 18.83 and a 
multiple R of 0.65. By including Question 3 in the second 
step and Question 9 in the third step, a cumulative multiple R 
of 0.69 was attained. This R represents 48% of the variation 
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in mean SOF Item 12 scores being explained by the three 
variables, curriculum of students, number of students in the 
class, and number of years the professor has been on the 
NFS faculty. The fourth and fifth variables to be entered 
in the regression analysis. Questions 11 and 16, respectively, 
were not included in the final regression equation because 
of their low statistical significance (both F values less 
than 2.0). 

After both regressions were run, the corresponding two 
regression equations were constructed using the unstandardized 
regression coefficients (B's) and the constants that were 
computed in the analysis. The variables predicted from these 
equations were labeled "PSOF" (predicted SOF) , as these were 
the mean SOF values that could be predicted for each pro¬ 
fessor based on his/her responses to the selected questions 
from the survey. Another variable, "RESSOF” (residual SOF), 
was also calculated by subtracting each professor's pre¬ 
dicted mean SOF value (PSOF) from his/her actual SOF Item 12 
score. RESSOF indicates the amount by which the actual mean 
SOF Item 12 score exceeds or falls short of the mean pre¬ 
dicted SOF score. The RESSOF indicates a professor's unique 
contribution to his/her SOF score apart from what could have 
been predicted by the questionnaire. Table II provides 
the mean SOF Item 12, PSOF, and RESSOF scores for the 28 
cases that were used in the analysis. A positive RESSOF 
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identifies those cases in which the professor received a 


higher mean 

SOF 

Item 12 score 

than was 

predicted by 

the 

regression. 

Conversely, a negative RESSOF indicates 

that a 

professor would 

have been expected to 

receive a higher mean 

SOF Item 12 

score than he/she 

actually 

did. 




Table II 





PSOF 

RESSOF 





Structural Only 





SOF 




Instructor 


Item 12 


PSOF 

RESSOF 

1 


4.550 


4.397 

.153 

2 


4.870 


4.775 

.095 

3 


4.800 


4.775 

.025 

4 


3.750 


4.217 

-.467 

5 


3.180 


4.413 

-1.233 

6 


3.950 


4.052 

-.102 

7 


4.000 


4.108 

- .108 

8 


3.380 


3.546 

-.166 

9 


4.190 


4.084 

.106 

10 


4.320 


4.084 

.236 

11 


4.090 


4.144 

-.054 

12 


3.640 


4.074 

-.434 

13 


4.000 


3.798 

.202 

14 


3.800 


3.798 

.002 

15 


5.000 


4.832 

.168 

16 


5.000 


4.790 

.210 

17 


4. 770 


4.371 

.399 

18 


4.950 


4.287 

.663 

19 


3.330 


3.858 

-.528 

20 


3.720 


3.928 

-.208 

21 


4.600 


4.238 

.362 

22 


4.170 


4.462 

-.292 

23 


4.680 


4.678 

.002 

24 


4.800 


4.497 

.303 

25 


4.060 


4.004 

.056 

26 


4.050 


4.494 

-.444 

27 


4.210 


3.756 

.454 

28 


4.030 


3.704 

.326 
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Structural and Subjective 


Instructor 

Item 12 

PSOF 

RESSOF 

1 

4.550 

4.470 

.080 

2 

4.870 

4.866 

.004 

3 

4.800 

4.866 

-.066 

4 

3.750 

4.025 

-. 275 

5 

3.180 

3.908 

-.728 

6 

3.950 

3.822 

.128 

7 

4.000 

3.822 

.178 

8 

3.380 

3.805 

-.425 

9 

4.190 

4.237 

- .047 

10 

4.320 

4.237 

.083 

11 

4.090 

4.478 

- .388 

12 

3.640 

4.197 

-.557 

13 

4.000 

3.641 

.359 

14 

3.800 

3.641 

.159 

15 

5.000 

4.920 

.080 

16 

5.000 

5.201 

-.201 

17 

4.770 

4.369 

.401 

18 

4.950 

4.665 

. 285 

19 

3.330 

3.796 

- .466 

20 

3.720 

3.796 

-.076 

21 

4.600 

4.267 

.333 

22 

4.170 

3.807 

. 363 

23 

4.680 

4.741 

- .061 

24 

4.800 

4.353 

.447 

25 

4.060 

4.033 

.027 

26 

4.050 

3.960 

.090 

27 

4.210 

4.029 

.181 

28 

4.030 

3.922 

.108 


The final analysis that was performed on these new 
variables was a Pearson correlation between the RESSOF and 
the actual mean SOF Item 12 score. This correlation repre¬ 
sents the strength of the linear relationship between these 
two variables, with possible values ranging from -1 for a 
perfect negative correlation to +1 indicating a perfect 
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positive correlation. A Pearson correlation of .58 was 
obtained for the analysis in which both structural and 
subjective variables were used, and .73 when only the 
structural variables were entered. These figures indicate 
how much the mean SOF Item 12 scores reflect the individual 
contribution of a professor, plus error of measurement. In 
the correlation which includes both structural and subjective 
variables, the .58 signifies that, at most, 34% of the 
variance in the SOF Item 12 scores is attributable to a 
professor's unique contribution. Likewise, in the analysis 
of the solely structural variables, at most, 53% of the 
variance in the SOF Item 12 scores reflects the professor's 
individual qualities. 

D. RESULTS AND APPLICATIONS 

The analysis of the professor questionnaire produced 
some interesting and potentially far-reaching results. The 
relative variability of the PSOF (predicted SOF) variable, 
or mean SOF Item 12 score that could be predicted based on 
the professor's responses to the survey, has some important 
implications. The finding that 67% of the differences among 
professors' mean SOF Item 12 scores are attributable to 
factors not related to teaching effectiveness or the quality 
of the course diminishes the credence that can be placed in 
the SOF as a measurement tool. Royce (1956) found in his 
study that instructors who are entertainers in class. 
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although their teaching quality may be mediocre, tend to 
receive higher student-faculty evaluations than those who 
are not so humorous or personable but who may be extremely 
effective teachers. [Ref. 25] The inclusion of Question 8, 
"How well do you think the students generally like this 
course?", in the second step of the first regression could 
be partially explained by this tendency. Another potential 
contribution to the strength of this variable (SOF Item 12) 
may be how enthusiastic, involved, and interested the pro¬ 
fessor is in the course. Guthrie (1954) found this to be a 
factor in how fond the students were of particular instruc¬ 
tors. [Ref. 26] 

The inclusion of Question 4 (curriculum membership of 
the class majority) in the first step of the regression 
implies a difference in SOF scoring based on discipline. 
Curriculum 857, Organizational Effectiveness, has the 
highest mean SOF Item 12 scores of all the curricula 
surveyed (a total of 23). Can this finding be attributed 
to the superiority of professors in that discipline? Is it 
a difference in their teaching approach or style? Perhaps 
students in that field are, by nature, more lenient 
evaluators. In any case, it appears that there are trends 
or similar tendencies among students in specific curricula. 

Another variable (the fourth) that was important to the 
variances in Item 12 mean scores, subject area, is not 
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surprising. Certain subject areas seem to be universally 
more attractive to students than others. In this study, 
the professors teaching Logistics/Material Management courses 
received the highest mean SOF Item 12 scores, at 4.68, with 
instructors in Financial Management courses following, with 
a mean of 4.38. Information Systems and Economics/Accounting 
categories received the lowest mean Item 12 ratings, at 3.8 
and 4.0, respectively. 

The fifth variable entered in the first regression, 

"Have you ever taught this student group before this quarter?", 
is also a reasonable factor in overall professor ratings. The 
effect of this variable could be directly or inversely 
related to the professor’s evaluation depending on whether 
the students had had positive or negative experiences in 
their previous classes with the professor. 

In the "structural only" regression, the number of students 
in the class was strongly related to the mean SOF Item 12 
rating a professor received (r = .35). This finding is in 
agreement with many studies on this subject. It has been 
suggested that students prefer small classes (and there¬ 
fore rate the professors higher) because they permit 
greater student-teacher interaction. 

Finally, tenure and number of years on the NFS staff 
were also determinants in student-faculty evaluations. 

Diverse results have been obtained by the many studies on 
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this subject. Although Heilman and Armentrout [Ref. 27] 
found no significant relationship between experience and 
student ratings, Downie [Ref. 28] found that full professors 
received higher ratings than did instructors in lower 
academic ranks. This was true especially on traits such as 
sense of humor, broad interests, and effective presentation 
of subject matter. 

In addition to the statistical analysis previously dis¬ 
cussed, it was decided that cross tabulations should be per¬ 
formed on a few of the variables, just as a matter of 
interest. The amount of weight an instructor thought the 
SOF should carry in pay, promotion, and tenure decisions was 
crosstabulated by the academic rank of the professor. One 
hundred percent of the professors, associates, and assistants 
believed that less than 40% of these decisions should be 
based on SOF score. Forty-five percent of the adjuncts 
believed less than 20%, whereas 55% thought between 41% and 
60% of the weight in management decisions should be based 
on SOF ratings. Of the military instructors, half selected 
less than 20% and half between 61% and 80%. Could it be that 
those with more experience with NFS SOF’s have learned (or 
developed the opinion) that they are not particularly 
valid or reliable performance measures and should not be 
trusted in such important career decisions? 

Another crosstabulation was executed utilizing the 
variables, "How well do you think the students generally 
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like this course?’* and "How well do you like teaching this 
course?" Based on this analysis, it appears that the pro¬ 
fessors, in general, like teaching the courses more than 
they think students like taking them! 

The results of these analyses can be used in various 
ways by NPS faculty and administrators, and hopefully they 
will shed some light on the myriad concerns in regard to the 
use of SOF's. Perhaps, as a result of this research, the 
school will have increased awareness of the numerous possible 
discrepancies and the many seemingly unrelated factors that 
actually have a considerable impact on student-professor 
evaluations. This study strongly indicates that great 
caution be exercised when using the SOF's as a major or 
exclusive determinant in important decisions such as pay, 
promotion, and tenure. 

If the Student Opinion Forms are to be heavily weighted 
in significant career decisions, the research results reported 
here suggest that a professor's ratings may be affected to 
a considerable extent by the subject area of the course and 
the student curriculum group that he/she is assigned to 
instruct, as well as the number of students in the class. 
Because the department chairman is responsible for the 
assignment of professors to courses and class sections and 
can also govern class size, he/she could have a substantial 
influence on increasing or decreasing a professor's SOF 
scores. 
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Another more direct use of the professor questionnaire 
data would be to review the specific responses of particular 
professors to the attitude questions. The extent to which 
they do not enjoy instructing their courses, have good 
rapport with their students, or feel SOF's improve their 
own teaching effectiveness may indicate some attitude 
problems that may be responsible, at least in part, for 
poor student-professor evaluations. 
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III. STUDENT QUESTIONNAIRE 


A. MOTIVATION 

In conjunction with the interest in the professors* 
viewpoints on SOF's and their effects and uses, attention was 
also directed toward the students' perceptions. In day-to- 
day conversation with students at NFS, it became apparent 
that there was a great lack of understanding about the use 
and importance of the SOF. Students related a multitude of 
different experiences they had encountered during the 
administration of SOF's at the end of their various courses. 
There were numerous comments/complaints about not having 
enough time to complete the SOF's and about the professors* 
wandering around the classrooms looking over students * 
shoulders. Concern was also expressed about how a student's 
grade may be affected if the professor were to see the SOF 
prior to the determination of final course grades. There 
was noticeable apathy among the students regarding the wasted 
effort expended in the completion of the SOF. Many students 
seemed to feel that no one really looked at the SOF's and 
that they served no function in management decisions. 

B. APPROACH 

The student questionnaire was distributed to 258 
Administrative Sciences students from 23 different curricula. 
They were completed in class and collected by the researchers 
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to attain a 100% return rate. The survey contained 20 
questions which were answered on a computer formatted answer 
sheet. The data were processed and collated by McGraw-Hill, 
Inc., who also provided basic summary statistics. 

It seemed important to obtain the students’ opinions, 
attitudes, and understanding of the Naval Postgraduate 
School’s Student Opinion Form in an effort to assess the 
reliability of the SOF data. The questionnaire items covered 
personal topics such as age, rank, service, and commission¬ 
ing source. Also included were opinion questions regarding 
factors that influence teaching effectiveness, as well as 
basic knowledge questions about the importance and use of 
the SOF. Appendix E lists the 20 items from the student 
questionnaire, the item response options, and the frequencies 
with which the students responded to each option. Below 
is a summary of the frequency analysis: 

* The majority of students completing the student 
questionnaire were between the ages of 28 and 36. 

* Current rank ranged from 02 to 06, the majority 
falling into 03 and 04--approximately 86%. 

* Commissioning sources varied. 

* The majority of students had served between eight 
and eleven years on active military duty (congruent 
with rank distribution). 

* Completion date of baccalaureate degree varied 
from three to 20 years ago, with the majority 
falling in the eight to eleven year range. 

* Approximately 70% of the students completing the 
questionnaire were serving in the U.S. Navy. 
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* Approximately 70% of the students found classroom 

layout and seating arrangement at least reasonably important 
for effective learning. 

* Only 10% of the students surveyed understood that the 
primary purpose and use of the SOF was for pay, promo¬ 
tion, and tenure decisions. One-third felt that the 
primary use of the SOF was to provide feedback to the 
professor, only, to help improve teaching effectiveness. 
Nearly 35% of the students thought the primary SOF use 
was to give the department chairman an idea of the pro¬ 
fessor’s performance or popularity. Thirteen percent 
had no idea what the forms were used for. 

* Fifty percent of the students felt that the SOF 
should be for use by the department chairman to evaluate 
the professor, in conjunction with other performance 
measures. Three percent felt the SOF’s should not be 
used at all. 

* Nearly three-fourths of the students felt that SOF’s 
should carry no more than 40% of the weight in pay, pro¬ 
motion, and tenure decisions. 

* For 57% of the students, the primary source of know¬ 
ledge concerning the use of the SOF was either an 
individual professor or other students. Twenty-one 
percent identified no source of information. 

* Most students (26%) felt that a written narrative 
evaluation of a professor by each student would most 
increase teaching effectiveness. Twenty-four percent 
responded that class visitation by department chair¬ 
man or other faculty would produce the best results. 

Only 14% thought that the current SOF was the most 
effective evaluation method. 

* Forty-two percent of the respondents believed that 
the professors never saw the SOF’s before grades were 
determined. Thirty-six percent had no idea. 

* Fifty-five percent of the students surveyed had always 
completed SOF’s before they knew their final grade. 

Forty-five percent had, on some occasions, already known 
their course grade. 

* Nearly half of the respondents had completed SOF’s 
outside the class at least once. 


51 


•t! 








* Sixty-five percent of those polled had com¬ 
pleted SOF’s in the presence of the professor. 

* Nearly half the students had, at one time or other, 
felt that they had not been given enough class time 
to complete the SOF. 

* Most students thought that the average score for all 
professors in the Administrative Sciences Department 
on SOF Item 12, the overall rating of the instructor, 
was between 3.6 and 4.5. 

* The majority of students believed that either the 
professor only, or both the professor and the depart¬ 
ment chairman, saw the comments on the back of the 
SOF. 

* Eighty percent of the students would have responded 
the same to the questions on the front of the SOF 

if they had known that only the professors saw the 
comments on the back. 

C. METHODOLOGY 

In addition to general interest in the basic frequencies 
of responses to the student questionnaires, the researchers 
thought that some other relevant findings might emerge 
from further statistical analysis. It was believed that 
there may be some tendencies, or patterns, to the way in 
which students responded to the survey questions based on 
such factors as rank, seniority in the curriculum (i.e., 
number of quarters completed), and branch of service. 

The most fruitful analysis of the student questionnaire 
data consisted of cross tabulations, or frequency breakouts 
of responses to one question according to the student's 
responses to another question. For the most part, in all 
of the cross tabulations performed, no clearcut division 
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of responses by seniority, rank, or branch of service was 
evident. All groups seemed to have similar tendencies in 
SOF understanding and beliefs, and they predominantly selected 
the same responses most frequently. There were, however, 
some interesting findings, which will be discussed below. 

In the crosstabulation between Question 8, understanding 
of the primary purpose and use of the SOF, and number of 
quarters completed, the majority of all students (regard¬ 
less of seniority) selected options 3 and 4. These were 
"feedback to the professor, only" and "to give the department 
chairman an idea of the professor's performance or popu¬ 
larity," respectively. Only those students in their fifth or 
sixth quarters seemed to realize that SOF's play an important 
part in pay, promotion, and tenure decisions. Those students 
in their first quarter were, understandably, more likely to 
respond, "I don't know," (37%), than students in any other 
quarter. 

The crosstabulation between Question 11, primary source 
of knowledge concerning the use of the SOF, and number of 
quarters completed, again, exhibited no ordinal response 
differentiation according to seniority. As noted in the 
previous section, "the individual professors" and "other 
students" were the most frequent selections. It was of 
considerable interest to find, however, that more than 20% 
of third-quarter students, nearly 25% of second-quarter 
students, and 45% of the first-quarter students had no 
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source of SOF information. Sixth-quarter students were the 
only group who had received a considerable amount of infor¬ 
mation from the department chairman or academic associate. 

In review of the crosstabulation of rank with the 
responses to Question 10, **How much weight do you think 
the SOF’s should carry in pay, promotion, and tenure deci¬ 
sions?", it was interesting to note that officers of the 
rank 05 (Commanders or Lieutenant Colonels) were the only 
group who predominantly (45%) felt that SOF’s should carry 
no weight. The majority of students in other ranks believed 
it should be between one and forty percent. 

Lastly, the cross tabulation of Question 6, branch of 
service, with Question 12, "Which of the following evaluation 
methods do you feel would most increase teaching effective¬ 
ness?", yielded notable results. Of all the services 
responding, the U.S. Coast Guard and the U.S. Army were the 
most satisfied with the current SOF. The civilian/foreign 
military response group would be equally content with 
either "individual student-professor conferences" or 
"written narrative evaluations." Again, options 3 and 4, 
"written narrative evaluation" and "section-leader conference 
with department chairman," respectively, were the most 
popular responses across all services. 
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D. RESULTS AND APPLICATIONS 

The data that were obtained from this study strongly 
support the impressions that came across during informal 
conversations with Administrative Sciences students. This 
survey revealed that a common source of information regard¬ 
ing the use, purpose, and importance of SOF's is totally 
absent. This, in addition to the considerable dispersion of 
responses as to the understood primary purpose and use of 
the SOF, implies that students have no uniform comprehension 
of the use of this reporting process. Stevens (1978) 
reported that a total lack of student knowledge regarding 
student-faculty evaluations is not uncommon. [Ref. 29] In 
his study, several upper level students admitted that they 
did not know what the SFE was used for, but that they believed 
it served solely as feedback to the professors. It can also 
be seen in the NPS students' responses to Questions 13 through 
17 and Question 19 that no universal standards, guidelines, 
or requirements for the administration of SOF's were adhered 
to by the professors. It is, therefore, imperative that 
complete information regarding the SOF become general know¬ 
ledge not only among the students but among faculty members 
as well. 

The fact that only 14% of the respondents felt that the 
current SOF was the most effective of the evaluation methods 
listed for increasing teaching effectiveness suggests that 
this is a topic that bears further review. Since the majority 
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of students lack faith in the SOF as an evaluation tool, 
this may imply that a lack of sincerity and genuine effort 
is being exerted in the completion of SOF's. It is ludi¬ 
crous to suppose that students will put forth a concerted 
effort in honestly evaluating a professor if they have no 
idea what the evaluation is used for, who sees it, and the 
amount of weight it carries in management decisions --especially 
if they have no faith in its usefulness. 

Fifty percent of the students believed that other 
performance measures should be used in conjunction with the 
SOF by the department chairman to evaluate the professor. 

This finding indicates a feeling that the SOF is inadequate 
and incomplete. They suggest that a written narrative 
evaluation of the professor by each student would be the most 
beneficial method to increase teaching effectiveness, 
closely followed by class visitations by the department 
chairman or other faculty. NFS administrators as well as 
faculty should seriously consider these recommendations, 
as they may provide more comprehensive and impartial 
evaluations. 
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IV. ADDITIONAL SOF ITEMS 


A. INTRODUCTION 

The third and final section of this research project 
involved the addition of four questions to the standard NFS 
Student Opinion Form. Since the SOF's are disseminated 
quarterly to all NFS students, and the additional four 
questions requested information relating specifically to 
each course a student was taking, it was decided that the 
SOF would be an appropriate vehicle for asking these 
questions. There are four spaces provided on the SOF 
(Items 17-20) for additional questions. The four questions 
added were devised to reflect relationships of Item 12 re¬ 
sponses with students’ perceptions of the course and of the 
professor. A sample of approximately 560 provided responses 
to these items. Since it has become traditional to use 
Item 12 as an indicator or gauge of the mean responses to 
the other 15 items, Fearson correlations were performed 
between the four additional items and Item 12 only. Table III 
lists the four additional SOF items. 

B. MOTIVATION 

The responses the students provided to the additional 
SOF questions were expected to be highly related to their 
responses on the original SOF items (Items 1-16). In par¬ 
ticular, it was anticipated that a student who was expecting 


57 



• i 






-a: f.. 


m 


Table III 


Additional SOF Items 


17. IVhat is your anticipated grade for this course? 

18. What is your total cumulative grade point average at 
the Naval Postgraduate School? 

19. Prior to this quarter, what were your expectations 
of this course in regard to interest, challenge, and 
potential usefulness in your career? 

20. To what extent do you think SOF's are worth your time 
and effort to fill out? 


a high grade would be more likely to rate both the professor 
and the course higher than those expecting lower grades. It 
was also predicted that overall grade point average (GPA) 
would be correlated with the other item responses; however, 
the researchers would not venture to guess whether this 
correlation would be positive or negative. It seemed equally 
reasonable to suppose that the students with higher GPA's 
would tend to be either harsher graders, as a whole, or more 
lenient than those with lower GPA's. If students had very 
high expectations of the course and were disappointed, then 
they might express their disappointment with exceptionally 
low SOF ratings. It was assumed that, if the responses 
to Question 20, "To what extent do you think SOF's are worth 
your time and effort to fill out?", were predominantly low, 
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then the students were not spending much time and effort 
completing the SOF's. If this were the case, it would 
significantly decrease the validity of the SOF as an evalua¬ 
tion tool; and, therefore, SOF use could be extremely and 
unfairly detrimental to a professor's career. 

C. METHODOLOGY AND RESULTS 

The first statistical analysis consisted of the compu¬ 
tation of a Pearson correlation between each of the four 
additional questions and SOF Item 12 (overall rating of the 
instructor). As was stated previously, the Pearson corre¬ 
lation measures the linear relationship between two variables. 
Contrary to expectations, all four correlations were quite 
low. The correlation coefficient between anticipated grade 
(Question 17) and SOF Item 12 was .10. This result implies 
that there is little linear relationship between the grade a 
student expects to receive from a professor and the way he/ 
she rates the professor. Again, there has been considerable 
research performed in this area, with very diverse results. 

Both the Guthrie [Ref. 30] and the Cohen and Humphrey [Ref. 31] 
studies found that there was a high linear relationship 
between a student's expected grade and the rating given to 
the instructor. However, Rubenstein and Mitchell [Ref. 32] 
and Russel and Bendig [Ref. 33] found exactly the opposite, 
in consonance with the findings of this study, that little 
correlation existed. Hence, it can be seen that there is 

little agreement among researchers on this subject. 
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The lowest correlation obtained in the analysis, and the 
only negative one (-.07), was between Question 18 (cumulative 
GPA) and SOF Item 12. This finding discounted the hypothesis 
that students with high GPA's tend to rate professors 
differently from those with low GPA’s. Supporting this 
finding were the results of Guthrie (1954) , who concluded 
that students with higher GPA's do not rate professors any 
differently from students with lower GPA's. [Ref. 34] An 
analysis performed by the University of Wisconsin--Green Bay 
Office for Educational Development (1972), using data from 
the school's Course Comment Questionnaires, also found no 
correlation between overall GPA and course evaluations. 

[Ref. 35] 

The correlation coefficient for Question 19 (prior expec¬ 
tations of the course) and SOF Item 12 was .25, again indi¬ 
cating little (but certainly notable) correspondence. Finally, 
the correlation between Question 20 (SOF's worth the time 
and effort to fill out) and SOF Item 12 was also small, at 
.13. The results of these correlational analyses are 
encouraging in that they imply that the validity of the NPS 
SOF’s may not be highly contaminated by the potential biases 
represented in Questions 17 through 20. 

Crosstabulations were also carried out between Questions 17, 
18, and 19 and Item 12. This analysis was intended to show 
how the students who chose each option to these three questions 
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rated their instructors. As previously discussed, there were 
no definite patterns to the responses to Item 12 based on 
the options selected for the three questions (i.e., low 
correlations). Interestingly, of the eight students who were 
expecting a C+ or below in their courses (option 0 to Question 
17), six of them rated their instructors excellent (option 4 
to Item 12). In each response category for all three 
questions, the great majority of students rated their instruc¬ 
tors above average, excellent, or outstanding. Also worthy 
of mention is the fact that in each response category for 
questions 17, 18, and 19, the greatest number of students 
rated their professors outstanding, followed by the number 
who responded excellent. This trend continued in descending 
order, with the fewest students rating their professors 
poor. These statistics imply considerable inflation in the 
students* evaluations of NFS professors. 

Various other correlations were computed in an effort 
to test hypotheses about relationships between Item 12 and 
the other variables on the SOF. It was anticipated that the 
number of quarters a student had completed at NFS would have 
a bearing on the student's overall rating of an instructor. 

The correlation obtained was, in actuality, very low, at .07. 
This result reveals virtually no tendency of overall rating 
to vary with seniority. Another belief was that a relation¬ 
ship would exist between the overall rating of the instructor 
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and whether the course he/she was teaching was required or 
elective. The correlation obtained not only was low, but 
also negative (-.19). Thus, there was a slight tendency 
for students taking a course as an elective to rate instruc¬ 
tors more leniently than other students. Cohen and Humphreys, 
in a 1960 study, found evidence to support this finding that 
teachers of required courses received lower ratings than did 
teachers of elective courses. [Ref. 36] 

There were also correlations run between Item 12 and the 
other original items on the SOF. The majority of these 
variables were highly related to Item 12, as Read [Ref. 37] 
had shown in his study, with the correlations ranging from 
.41 to .77. (Question 16 was not considered because of the 
predominance of courses with no laboratory.) This high 
multicollinearity somewhat reduced the independence of the 
variables' explanatory power in the regressions reported 
below. 

Contrary to the expectations expressed earlier, there 
was very little association between the way students 
answered the additional four SOF items and their responses 
to the original 16 items. The highest correlation coefficient 
obtained was .39, but the majority were below .15. This 
result, again, supports the findings that expected grade, 

GPA, and expectations of the course do not determine the 
student's views of course and instructor effectiveness. It 
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also suggests, in conjunction with the results discussed 
in the previous paragraph, that SOF Item 12 is a good 
summary of the other questions. 

Several regression analyses were executed, all of them 
employing SOF Item 12 as the dependent variable. Because of 
numerous missing values, the automatic listwise deletion of 
the regression program reduced the sample size to 449 in 
these analyses. Initially, only the four additional SOF 
items (Questions 17-20) were regressed on Item 12, for the 
purpose of determining how much of the variation in Item 12 
was due to these items, as a group. The multiple R resulting 
from the inclusion of all four variables was 0.29, revealing 
that a very small portion of Item 12 variation (only 81) 
was possibly a result of attitudes or conditions reflected 
by these additional SOF questions. 

The second regression, again using SOF Item 12 as the 
criterion variable, included number of hours taken this 
quarter (HRS), number of quarters completed (QTRS), whether 
the course was required or elective (REQELEC), Questions 1-11, 
and Questions 17-20 as the battery of predictor variables. 
Though unreported here, various other combinations of predictor 
variables from the SOF were also regressed on Item 12, and 
comparable results were obtained. The other four "overall" 
variables (Questions 13-16) were not included in the 
regressions. 
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In this (the second) analysis, the stepwise regression 
was selected, limiting the number of variables to be included 
to five. Question 9, "The instructor made this course a 
worthwhile learning experience," was entered in step one 
with a multiple R of 0.77 and an F-value of 652. Step two 
included Question 3, "The instructor seemed to know when 
students did not understand the material," raising the 
multiple R to 0.82. Question 1, "The course was well 
organized," came in on step three, increasing the multiple R 
Only one more notch, to 0.83. Steps four and five included 
Question 10, "The instructor stimulated my interest in the 
subject area," and Question 11, "The instructor cared about 
student progress and did his share in helping us to learn," 
raising the multiple R, finally, to 0.84. Thus, 11 % (84^) 
of the diversity in overall instructor ratings is determined 
by these five variables. In other words, if the students 
felt the course was a worthwhile learning experience, the 
instructor was sensitive to the students' understanding of 
the material, the course had good organization, the instructor 
was able to interest the students in the subject, and the 
instructor genuinely cared about student progress and was 
instrumental in helping them learn, then they were quite 
likely to consider the professor an effective instructor. 
Finally, response frequencies were determined for all 
20 SOF questions to obtain an overall distribution of item 
responses for the Administrative Sciences Department. These 
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frequencies are contained in Appendix F. Below is a 
synopsis of the most noteworthy findings from the frequency 
analysis. 

* The mean response for Items 1 through 15 was 3.9. 

* The mean response for SOF Item 12 was 4.0. The range 
was from 2.6 to 5.0 (using mean SOF Item 12 values for 
each class). 

* All questions had a mode (most frequent response) of 
either 4 or 5, except Questions 16 and 20. 

* The items with the highest means were Question 6, 

*'1 felt free to ask questions,” (mean: 4.5); Question 5, 

"I had confidence in the instructor's knowledge of the 
subject,” (mean: 4.4); and Question 7, "The instructor 
was prepared for class," (mean: 4.3). 

* Eighty-four percent of the students were expecting 
a B+ or better in their courses. 

* Eighty percent of the students had a cumulative GPA 
of 3.25 or above. 

* Eighty-six percent of the students had moderate to 
very high expectations of their courses. 

* Forty-two percent of the students felt that SOF's 
were worth their time and effort to fill out only to a 
moderate extent. This was the response most frequently 
selected; and it was the cause of Question 20's having 
the lowest mean, 2.9, of all the SOF items (again, 
disregarding Question 16). 

As can be seen, there is much valuable analysis that can 
be done on the Student Opinion Form data. This study has 
only scratched the surface of the research potential in 
this area. It is very difficult to detect, measure, and 
analyze factors that may come to bear on a student's percep¬ 
tions of course and instructor quality. It was the researchers' 
intention to delve into four of these factors (Questions 17-20) 
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to determine if they might represent strong biases in the 
students* evaluations. The results obtained in this brief 
part of the study were gratifying in that they suggest that 
NPS students are able to minimize the effects of at least 
some extraneous factors that may impinge on their impartial 
evaluation of a professor's effectiveness. 
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V. CONCLUSIONS AND RECOMMENDATIONS 


A. CHARACTERISTICS OF SAMPLE 

The Naval Postgraduate School is considered to be a 
technical institution geared toward the Master o£ Science 
level. The curricula range from the "soft" sciences such as 
Administrative Sciences and National Security Affairs to the 
"hard" sciences such as the numerous engineering programs. 

The students at NPS are not, by many measures, typical 
graduate students. The median age is between 31 and 33, 
suggesting a higher maturity level than the average graduate 
student. The student body is comprised of primarily U.S. 
military officers, a considerable number of foreign military 
officers, and a handful of civilians. They are highly 
structured and disciplined individuals. Although the cur¬ 
ricula are all considered extremely rigorous, the highly 
motivated and competitive nature of the students drives them 
to excel. 

The Naval Postgraduate School's professors are recruited 
from some of the country's best schools. They are intrigued 
by the maturity, motivation, and dedication of the students, 
as well as the school's heavy emphasis on teaching. 

The samples used in this research do not claim to be 
representative of graduate students in general or of 
instructor populations outside the Naval Postgraduate School; 
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nor are they representative of the entire Naval Postgraduate 
School. 


B. OVERVIEW OF RESEARCH RESULTS 

The following is a recapitulation of the prominent 
results obtained in each study: 

1. Professor Questionnaire 

* Sixty-seven percent of the variation in SOF Item 12 
scores can be attributed to curriculum of the students, 
how well the professor thinks the students like the 
course, the subject of the course, and how well the 
professor likes teaching the course. 

* A professor’s ratings may be affected to a considerable 
extent by the department chairman in that he/she can 
decide the subject(s) and students groups that the 
professor teaches as well as the number of students that 
are in the class section. 

* The professors are not well informed as to the 
importance of SOF's in pay, promotion, and tenure 
decisions. 

* Most professors do not feel that SOF’s actually 
measure teaching effectiveness to a great extent. 

* Over one half of the professors surveyed considered 
SOF’s to be only somewhat useful in improving their 
teaching effectiveness. 

2. Student Questionnaire 

* It was found that very few students understood the 
purpose and use of the SOF, and its importance in pay, 
promotion, and tenure decisions. They were also unclear 
as to who was authorized access to the SOF information. 

* It was also found that the students have no common 
reliable source (if any) of information regarding the 
SOF. 

* A number of students surveyed felt that other evalua¬ 
tion methods (e.g., class visitation) should be used in 
conjunction with the current SOF to improve teaching 
effectiveness. 
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* The majority o£ students did not feel that the current 
SOF was the best method for increasing teaching 
effectiveness. 

* Student responses indicated a lack of comprehensive 
guidelines for SOF administration and completion, 

3. Additional SOF Questions 

* A student's anticipated grade for a course had a 
very low correlation with the overall rating he/she 
gave the professor (SOF Item 12). 

* A student's cumulative GPA had very little relation¬ 
ship with the overall rating he/she gave the professor. 

* Students generally tended to have high prior expec¬ 
tations of their courses in regard to interest, 
challenge, and potential usefulness in their careers. 

* The majority of students felt that SOF's were 
worth their time and effort to complete. 

Since this study was fairly limited in scope, the 

results should be considered only suggestive; nevertheless, 

they do seem to complement certain findings previously 

discussed. 


C. DISCUSSION OF SFE FACTORS 

Presently, the major defense for defining quality 
teaching in terms of high ratings on the student evaluation 
forms is based on an analogy between the student and the 
consumer. The student may be in the best position to 
evaluate the professor's teaching effectiveness since he/she 
is the main consumer of the professor's product, namely, 
education. In addition to this belief, the ease of distri¬ 
bution to the students, as well as the extent to which 
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student-faculty evaluation is already used in colleges and 
universities throughout the country, tends to justify their 
continued use as a primary method of enhancing teaching 
effectiveness, 

It is important that student views be widely solicited 
in the form of SFE, although no one suggests that students 
are an infallible judge of instructor competence. Caution 
should be exercised to avoid sole dependence of performance 
evaluation on quantifiable SFE’s. Frequently, these evalua¬ 
tions are considered more important and reliable than others, 
strictly because they are quantitative and can be processed 
by a computer. It should be noted, however, that a compre¬ 
hensive evaluation process entails the observation of 
multiple performance measures, weighing and balancing them 
against each other, and ranking their importance in terms of 
the goals and objectives of the institution. 

For many years there has been a nearly universal per¬ 
ception that publication output has been the primary factor 
in promotion decisions. Indeed, there has been found to be 
a high correlation between academic rank and publication 
output rate. This finding has been the subject of much 
criticism. Is it possible that the reward system within 
higher education requires a professor to '’publish or perish"? 
Certain observations definitely support this premise. Higher 
status in graduate schools is conferred upon accomplished 
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research-scholars rather than instructor-scholars. In 

reference to this belief, a student was quoted: 

I came to graduate school wanting to be a liberal 
arts teacher. I now want to do research. I consider 
this is a moral decline on my part but I have learned 
that research is where the money, the prestige and 
the mobility are. [Ref. 38] 

At smaller schools, new faculty, who see themselves as 
temporary employees, place great emphasis on research as a 
means of professional survival. The large universities, 
which do place primary emphasis on publication, exert a 
tremendous influence on those schools that do not. As a 
result of this influence, professors aspiring to eminence 
generally sense the pressure to publish, regardless of 
institutional affiliation. 

Although faculty administrators bear responsibility for 
knowledge of the staff's teaching ability, many obstacles 
exist to hinder this awareness. Administrators typically do 
not have the time to personally observe every professor's 
teaching capabilities, especially if the department is large. 
If the department chairman is able to make one or two class 
visits, this visitation may be sufficient to judge certain 
elements of teaching, but it is not basis to make a compre¬ 
hensive evaluation. It cannot be assumed that elements such 
as number of classes taught, class size, or number of thesis 
advisees necessarily correspond to high quality instruction. 
Although the majority of professors are uncomfortable with 
classroom visitations and resent being "watched over," 
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many administrators consider this technique a very important 
element in evaluation. However, the department chairman 
may not be the most qualified judge of effective teaching. 

In addition, viewpoints solicited from other faculty members 
within the department may be prejudiced, may represent an 
inadequate sample, and frequently may be merely hearsay. 

These inherent factors create serious weaknesses in a system 
that relies heavily on teaching evaluations, as traditionally 
made by the department chairman or other faculty members. 

Circumstances beyond the professor's control may 
adversely affect his/her student ratings. Conditions such 
as heavy workload, large classes, being assigned to teach 
courses that are not in his/her area of professional expertise, 
or teaching a newly designed or particularly difficult course 
may greatly influence the evaluations a professor receives. 
Likewise, characteristics that are peculiar to the individual 
student may influence the evaluation. Variables such as the 
student's natural attraction to the subject matter, his/her 
need to compensate for poor grades, or whether the course 
is required or an elective may also have an effect. It has 
also been found that students' evaluations tend to reflect 
the personal and social characteristics of an instructor, "who 
he is" rather than "what he does." [Ref. 39] 

On occasion, situations occur in which there exists a 
severe personality conflict between a student and a pro¬ 
fessor. In cases such as this, a professor may receive an 
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unjustly low student rating based on this student's hostility, 
a rating in no way related to his/her teaching quality. 

A few very low scores may severely lower a good professor's 
mean rating, and may well be a detriment to his/her career. 

Another point of concern is the handling of "N/A" or 
"No Comment" responses. If they are not disregarded or 
treated as missing values, they may inadvertently penalize the 
professor. Therefore, administrators should be encouraged 
to assess the total distribution of responses to student- 
faculty evaluation items, and not just the mean overall 
score. Unfortunately, many such factors that are beyond 
the professor's control are not taken into consideration; and 
the professor is frequently penalized unfairly because of 
them. As a result, the professor may be forced to offer 
only "safe and familiar" instruction. 

Students evaluate a course based on their present 
experience, but do not look at the potential benefits that 
may be realized at a later date. It has been found that 
students, if given another opportunity later to rate pro¬ 
fessors, would give them higher scores now than they did 
when they took the course. 

In conclusion, the above circumstances, which are 
inherent in any educational system, should be considered 
potentially problematic; and, when they exist, they should be 
brought to the attention of school administrators. 
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D. RECOMMENDATIONS 


The following recommendations are derived from the results 
of this study, as well as from suggestions solicited from 
students, faculty, and administrators. 

1. This study has suggested a definite need for a more 
comprehensive explanation of the use and purpose of the SOF. 
Information regarding the SOF's role in pay, promotion, 

and tenure decisions should be widely publicized. A 
simple remedy might be the presentation of this infor¬ 
mation at a quarterly Superintendent's Lecture which is 
mandatory for all new students. Another possibility 
might be the dissemination of this information at a 
lower level, such as department or curriculum indoctri¬ 
nations or "Welcome Aboard" meetings or printed on the 
SOF itself. 

2. Standardized instructions for the completion of the 
SOF must be reiterated every quarter to ensure compliance 
and to alleviate the current serious editing problem, 

3. In order to collect a larger and more representative 
sample, questionnaires similar to the ones used in this 
research should be administered to the entire NFS faculty 
and student body. Additionally, further in-depth analysis 
of SOF data is called for. There is a vast amount of 
potential analysis that has yet to be tapped in this area. 

4. Information that the SOF provides would be considerably 
more helpful to professors than currently if they were 
counselled upon receipt of this feedback by the depart¬ 
ment chairman or another, knowledgeable faculty member. 

With counselling, negative feedback and criticism can result 
in positive action rather than a debilitating emotional 
issue. 

5. It is imperative that the professors be fully aware 
of the implications of the ratings received on the SOF's. 

If there is no differentiation in the action taken by the 
administration when a professor receives a 3.0 rating as 
opposed to a 5.0, then there may be no incentive for the 
professor to pursue excellence in teaching. There must 
be a reward or incentive system that is directly related 
to the evaluation system. 
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6. It would be very beneficial to have an informal 
midquarter evaluation to be used as an early indicator 
of the students' initial impressions of the quality of 
the professor's teaching. This evaluation would only be 
seen by the professor, and would allow time for cor¬ 
rections and improvements to be made during the same quarter 
so that the current class could benefit from their own 
suggestions. A recommended format for this midquarter 
evaluation would be a written narrative evaluation from 
each student. This evaluation would pinpoint specific 
problems that may exist, and would provide a much 

more personal form of feedback. 

7. Results from the student questionnaire revealed that 
students prefer the use of alternative forms of teacher 
evaluation in conjunction with the current SOF. More 
specifically, they suggest class visitation by the 
department chairman or other faculty member, a written 
narrative evaluation, and videotaping of class sessions. 

If class visitation is simply not feasible because of 
time constraints, then videotaping might be a preferred 
alternative, as some departments already possess audio¬ 
visual equipment. 

8. The handling of "N/A" and "No Comment" responses 
should be investigated to ensure that the professors 
are not being penalized by them. Since these responses 
are treated as "0" on a "0" to "5" integer scale, then 
their inclusion in the statistical computation or mean 
scores would considerably decrease the professors' 
ratings. 

9. The rewriting of the current SOF to be a more 
flexible and comprehensive evaluation tool than it now is 
should be seriously considered. The current SOF is 
regarded by some as too structured because all the 
administration sees is the 16 questions on the front; 
and these questions do not necessarily cover every facet 
of teaching effectiveness. For example, where is exces¬ 
sive homework or a professor’s condescending attitude 
indicated on the SOF? An evaluation form that is sub¬ 
divided into different categories should be constructed. 

The professor would be evaluated only on those factors 
over which he/she has control, and which are not 
vulnerable to the student's individual idiosyncracies. 

For instance, an item such as how well the student 

likes the subject is a matter of personal preference 
over which the professor has limited control. Addition¬ 
ally, the current heavy reliance on the professor’s 
overall rating (SOF Item 12) can be dangerous in that this 
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may be only a manifestation of the student's emotional 
reactions to the entire course. Much of it is not related 
to the professor or his/her teaching, or more importantly, 
to the amount the student has learned. 

10. It has been suggested that the last question on the 
SOF read, "Do you have any comments that you would 
particularly like the department chairman to see?" 

If the student responded yes to this question, then it 
would prompt the department chairman to read the 
"comments" section. 

11. It has also been suggested that the number of 
options for each SOF item be reduced from five to 
three (in addition to the "N/A" and "No Comments" 
options), as most students are reticent to rate a 
professor one or two (i.e., poor or fair). 

12. Another recommendation is that two forms of 
teaching evaluation be instituted at NFS, one strictly 
to be used as feedback to the professor (qualitative), 
and a second for use in administrative decision 
making (quantitative). 

13. In addition to being quantitatively evaluated on 
teaching proficiency, a professor should also receive 
a comparable evaluation for his/her research and pub¬ 
lication efforts. 

14. As a final note, it is imperative that all elements 
of a professor's work--teaching performance, research, 
and publication--be taken into consideration when 
making administrative decisions. 
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APPENDIX A 


STUDENT OPINION FORJvl 


STUDENT OPINION FORM 

UNO NPS 5040/2 (REV 5-83) 


INSTRUCTOR NAME- 


PLEASE USE SOFT BLACK PENCIL ONLY — DO NOT USE INK. | 



CURSMCIALTY 

NUMBER 


0 0 0 
0 0 © 
0 0 
0 0 
0 0 
0 0 
0 0 
0 0 
0 0 
0 0 


HRS THIS 
OTR 


0 0 
0 0 
0 0 
0 0 
0 0 
0 0 
0 0 
0 0 
0 0 
0 0 


QTRS 

COMPro 


0 0 
0 0 
0 0 
0 0 
0 0 
0 0 
0 0 
0 0 
0 0 
0 0 


FOR ME THIS 


COURSE IS: 


REQUIRED.... 

0 

ELECTIVE. 

0 



2 

o£ 

20 


IS 


©000 
0 0 0 0 


0 

0 

0 

0 


12. Overall, i would rate this instructor. 

13. Overall. I would rate this course. 

14 Overall. I would rate the texttX}ok(s) . 

15. Overall, t would rate the quality of the exams . 

16. Overall. I would raie the taboretories. 


0 0 0 
0 0 0 
0 0 0 
0 0 0 
0 0 0 


0 0 0 
0 0 0 
0 0 
0 0 


0 

0 


oo 

(AO 2U 


0 
0 
0 

0 0 0 0 


1. The course was well orgenued. 0 0 0 0 0 

2. Time in class wes spent effectively. 0 

3. The instructor seemed to know when students didn't understand the metenel. 0 

4. Difficult concepts were made understandable. 0 0 

5 1 had confidence m the instructor’s knowledge of the subiect. 0 0 -0 0 

8. I felt free to ask questions. 0 0 0 0 

7. The instructor was prepared for class. 0 0 0 0 

8. The instructor s objectives for the course have been made deer. 0 0 0 0 

9. The instructor made ihis course a worthwhile learning experience. 0 0 0 0 0 0 

10. The instructor stimulated my interest in the subject area. 0 0 0 0 0 0 

11. The instructor cared about student progress and did his share m helping us to leern . 0 0 0 0 0 0 

PLEASE USE THE FOLLOWING SCALE FOR THE NEXT FIVE ITEMS: 

5. Outstanding (Among ttx too tO%l 2. Fair (In tha lowaat 30%) 

A. Eicailant (Among tha top 30%l t. Poor (In tha lowaat tO%i 

3. About Avaraga (MiddlaAOS) O. Not AopDcAOta/Don't know/Thafa wara nona n P a F o na 


0 

0 

0 

0 


0 0 0 


USE SPACE BELOW ANO ON REVERSE StOE FOR FREE 
FORM COMMENTS. lOENTlFY BY QUESTION NUMBER 
WHEN APPROPRIATE. These free form comments will be 
available only to the initructor. 


THESE FOUR SPACES 
ARE AVAILABLE IF THE 
INSTRUCTOR WISHES 
TO SUPPLY AOOmONAL 
QUESTIONS. 


170 0 0 

18.0 0 0 
19.0 0 0 

20.0 0 0 


0 0 0 
0 0 0 
0 0 
0 0 


0 

0 


4529-OOP-5A321 
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APPENDIX B 


STUDENT COUNCIL MEMORANDUM 


MAY 78 

STUDENT OPINION FORM 


The Student Council would like you to know the following facts 
about the Student Opinion Form (SOF) ; 

What is the SOF? 


The SOF is the machine-readable form filled out by each student 
at the end of each quarter in each course. 

Is the SOF important? 

The SOFs (in summary form) are used by the Department Chairmen 
to assist in identifying Faculty members for pay raises and 
in tenure considerations. I'he suFs are used by the individual 

instructor to improve individual teaching techniques and 
improving course material. 

How does the SOF system work? 

Each student fills out a form in every course. The forms are 
collected and machine read, storing the total data. A 
statistical report is formed from this data for each pro¬ 
fessor and is forwarded to his Department Chairman. The 
Chairman uses this statistical data to assist in the 
evaluation of the teaching ability of each professor. 

The original form with its free hand comments intact is 
returned to the professor concerned. 

What are the problems? 


Casual preparation is a major problem which can be corrected by 
the students. Such seemingly inconsequential errors as 
illegibility, not using a #2 pencil, failure to complete 
all blocks or answer all questions, not to mention flippant 
or insincere remarks, all degrade the worth of the form both 
to instructors and to the Department Chairman. Insufficient 
time for preparation of the form is a problem which should 
be a matter of concern both for section leaders and pro¬ 
fessors. Cooperation is the key here. The correct inter¬ 
pretation of the "Quarters Completed" block is to include 
the present quarter as one completed. 
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What has been learned from the SOF data analyzed to date? 


From the small amount of useful data, the following trends 
are clear: 

a. Students have high confidence in the instructors' 
knowledge of the subject area. 

b. Students have a universally low opinion of the 
examinations. 

c. There does not appear to be a correlation between 
the grades given for the course and the rating of the pro¬ 
fessor by the students. 

d. In general, the professors who get the "best" marks 
from the students are the ones who teach the most "popular" 
courses. 


THE STUDENT COUNCIL BELIEVES THAT THE STUDENT VOICE IN THE 
ADMINISTRATION OF THE FACULTY (IN THE FORM OF THE SOF) SHOULD 
BE SUSTAINED AND THAT IT IS THE STUDENT'S RESPONSIBILITY TO 
GIVE THIS IMPORTANT FEEDBACK VEHICLE THE ATTENTION THAT IT 
DESERVES. IF WE RENDER A CASUAL OPINION, IT WILL COME TO 
BE CASUALLY REGARDED. 


The Student Council welcomes your comments through your 
Representative. 
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APPENDIX C 


SOF RETURN RATE 


ADMINISTRATIVE SCIENCES DEPARTMENT 
SOF RETURN RATE TABLES 



Q1,AY'77 

Q2,AY'77 

Q3,AY’77 

Q4,AY’77 

TOTAL SENT 

672 

947 

788 

734 

NO. RETURNED 

505 

760 

488 

397 

RETURN RATE % 

75% 

00 

o 

62% 

54% 



Q1,AY'78 

Q2,AY'78 

Q3,AY'78 

Q4,AY'78 

TOTAL SENT 

803 

833 

963 

825 

NO. RETURNED 

600 

613 

807 

627 

RETURN RATE % 

75% 

74% 

84% 

76% 



Q1,AY'79 

Q2,AY'79 

Q3,AY'79 

Q4,AY'79 

TOTAL SENT 

910 

1023 

1037 

1030 

NO. RETURNED 

727 

834 

798 

638 

RETURN RATE % 

80% 

82% 

77% 

62% 



Q1,AY'80 

Q2,AY'80 

Q3,AY'80 

Q4,AY'80 

TOTAL SENT 

976 

1009 

982 

820 

NO. RETURNED 

762 

664 

715 

532 

RETURN RATE % 

78% 

66% 

73% 

65% 
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Q1,AY‘81 

Q2,AY’81 

Q3,AY'81 

Q4,AY’81 

TOTAL SENT 

964 

774 

768 

707 

NO. RETURNED 

674 

525 

661 

512 

RETURN RATE 1 

70% 

68% 

86% 

72% 



Q1,AY’82 

Q2,AY’82 

Q3,AY'82 

Q4,AY’82 

TOTAL SENT 

853 

955 

937 

848 

NO. RETURNED 

586 

703 

784 

700 

RETURN RATE % 

69% 

74% 

00 

83% 



Q1,AY‘83 

Q2,AY’83 

Q3,AY’83 

Q4,AY’83 

TOTAL SENT 

1125 

1212 

1111 

942 

NO. RETURNED 

973 

911 

986 

854 

RETURN RATE % 

86% 

75% 

89% 

91% 



Q1,AY'84 

Q2,AY'84 

TOTAL SENT 

899 

1252 

NO. RETURNED 

778 

1057 

RETURN RATE % 

87% 

84% 
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APPENDIX D 


PROFESSOR SOF QUESTIONNAIRE 


The following questionnaire is part of a thesis project that 
will provide information on the administration and use of 
Student Opinion Forms. This questionnaire will be used in 
conjunction with an associated questionnaire which has been 
distributed to Administrative Sciences students. As this 
questionnaire will be used strictly for research purposes, 
we request that you do not identify yourself on this form. 


We would greatly appreciate your objective responses to the 
following twenty-three questions. Please circle your responses 
or fill in the Ijlanks, as appropriate. A separate question¬ 
naire should be completed for each class segment that you 
are teaching this quarter. 


1 . 


What 

16% 

21 % 

16% 

0 % 

11 % 

16% 

10 % 

10 % 


is the general subject area of this course? 

(a) Manpower/Personnel Management 

(b) Financial Management 

(c) Organizational Effectiveness 

(d) Probability/Statistics/Operations Research 

(e) Information Systems 

(f) Economics/Accounting 

(g) Logistics/Material Management 

(h) Other 


2 . 


What 

13% 

87% 


is the academic level of this course? 

(a) 1000-2000 

(b) 3000 

(c) 4000 


3. How many students are enrolled in this class? 


Ranged from 5 to 46 

4. What is the curriculum number (e.g., 847) of the majority 
of this class? 


360, 365, 366, 367, 620, 815, 827, 837, 847, and 857 

5. Do you know who the section leader or senior officer in 
this class is? 

24% (a) Yes 

76% (b) No (If no, then go to question #8.) 
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6 . 


To what extent do you feel the section leader or senior 
officer has influence over the class in relation to the 
course or the instructor? 


0% 

(a) 

Very great extent 

3% 

(b) 

Large extent 

29% 

(c) 

Moderate extent 

13% 

(d) 

Small extent 

18% 

(e) 

Very small extent 

11% 

(f) 

No extent 

26% 

(g) 

N/A 


7. Gradewise, what quartile do you expect the section leader 
or senior officer to fall in? 

24% (a) Top 

34% (b) Second 

16% (c) Third 

2% (d) Bottom 

24% (e) N/A 

8. How well do you think the students generally like this 
course? 

8% (a) Extremely well 

32% (b) Very well 

45% (c) Fairly well 

13% (d) Not very well 

0% (e) Not at all 


9. How many years have you been on the staff at the 
Naval Postgraduate School? 

18% (a) Less than one year 

40% (b) One to three years 

13% (c) Three to five years 

5% (d) Five to seven years 

24% (e) Seven or more years 


10 . 


What is your academic rank? 

18% (a) Professor 

16% (b) Associate 

21% (c) Assistant 

29% (d) Adjunct 

16% (e) Military Instructor 


11. Do you have tenure? 

24% (a) Yes 

(b) No 

(c) Not applicable 


76% 
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12. What step are you presently on the faculty salary schedule? 


0% 

(a) 

1-5 

21% 

(e) 

21-25 

8% 

(i) 

41-45 

0% 

(b) 

6-10 

2% 

(f) 

26-30 

11% 

(j) 

46-50 

0% 

(c) 

11-15 

11% 

(g) 

31-35 

2% 

(k) 

51-55 

0% 

(d) 

16-20 

13% 

(h) 

36-40 

0% 

(1) 

56-60 


32 % missing 


13. To what extent do you consider this course to be in your 
area of professional expertise? 

61% (a) Very great extent 

18% (b) Large extent 

11% (c) Moderate extent 

5% (d) Small extent 

5% (e) No extent 

14. How many courses (exclusive of labs) are you teaching 


this 

quarter? 

21% 

(a) One 

63% 

(b) Two 

16% 

(c) Three 

0% 

(d) Four or more 


15. How many times have you taught this course prior to 
this quarter? 

24% (a) Zero 

25% (b) One 

11% (c) Two 

11% (d) Three 

29% (e) Four or more 

16. Have you ever taught this student group (plus or minus 
a few students) before this quarter? 

24% (a) Yes 

76% (b) No 

17. How well do you like teaching this course? 

37% (a) Extremely well 

45% (b) Very well 

13% (c) Fairly well 

5% (d) Somewhat well 

0% (e) Not at all 

18. To what extent do you feel that you have good rapport 

with this class? 

34% (a) Very great extent 

45% (b) Large extent 

18% (c) Moderate extent 

3% (d) Small extent 

0% (e) Very small extent 
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19. 


Do you tend to organize and present this course 
differently according to the seniority of the student 
group that you are teaching (e.g., first-quarter vs. 
last-quarter thesis students)? 

24% (a) Yes 

37% Cb) No 

39% (c) Not applicable 

20. How much weight (percentage-wise) do you think the 
SOF forms presently carry in pay, promotion, and 
tenure decisTdnsT 


29% 

(a) 

0-20% 

32% 

(b) 

21-40% 

26% 

(c) 

41-60% 

0% 

(d) 

61-80% 

0% 

(e) 

81-100% 

13% 

(f) 

No idea 


21. How much weight (percentage-wise) do you think the SOF 
forms should carry in pay, promotion, and tenure 
decisions 'i 

40% (a) 0-20% 

34% (b) 21-40% 

16% (c) 41-60% 

10% (d) 61-80% 

0% (e) 81-100% 

22. To what extent do you feel SOF's actually measure 
teaching effectiveness? 

3% (a) Very large extent 

10% (b) Large extent 

45% (c) Moderate extent 

37% (d) Small extent 

5% (e) Very small extent 

23. How useful do you feel the SOF is in improving your 
teaching effectiveness? 


3% 

(a) 

Extremely useful 

13% 

(b) 

Very useful 

66% 

Cc) 

Somewhat useful 

5% 

(d) 

Not very useful 

13% 

(e) 

Not useful at all 


SAMPLE SIZE = 38 
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APPENDIX E 


STUDENT SOF QUESTIONNAIRE 


The following questionnaire is part of a thesis project that 
will provide information on the administration and use of 
Student Opinion Forms. As this questionnaire will be used 
strictly for research purposes, we request that you do not 
identify yourself on the answer sheet. We will be using the 
SOF as the answer sheet for this questionnaire. 

Please complete the following boxes at the top of the SOF 
answer sheet: 

Cur-Specialty Number (e.g., 847) 

Hrs This Quarter 

Qtrs Completed (including this quarter) 

Disregard the preprinted SOF question and answer the follow¬ 
ing twenty questions in the appropriately numbered spaces 
on the SOF answer sheet. Please fill in one response for 
each question. 

We greatly appreciate your time and effort in assisting us 
in our research. 


What 

is your present age? 



6% 

(5) 

27 or younger 



24% 

(4) 

28-30 



28% 

(3) 

31-33 



24% 

(2) 

34-36 



14% 

(1) 

37-39 



4% 

(0) 

40 or older 



What 

is your current rank? 



0% 

(5) 

01 



4% 

(4) 

02 



45% 

(3) 

03 



41% 

(2) 

04 



8% 

(1) 

05 



2% 

(0) 

Civilian 



What 

is your commissioning source? 



17% 

(5) 

One of the U.S. military 

academies 

or the Citadel 

21% 

(4) 

Reserve Officer Training 

Corps (ROTC) 

44% 

(3) 

Officer Candidate School 

or Officer 

Training 



School (OCS, OTS) or NESEP or AOCS 


3% 

(2) 

Officer Indoctrination School (OIS) 

or 


direct commission 
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4. 


5. 


6 . 


13% (1) Foreign military 

2% (0) Not applicable 

How many years have you served on full-time active 


military 

duty? 

1% 

(5) 

0-3 

25% 

(4) 

4-7 

40% 

(3) 

8-11 

2 2% 

(2) 

12-15 

9% 

CD 

16-19 

3% 

(0) 

20 or more 


How many years ago did you receive your baccalaureate 
degree? 


1% 

(5) 

0-3 

24% 

(4) 

4-7 

41% 

(3) 

8-11 

24% 

(2) 

12-15 

9% 

(1) 

16-19 

1% 

(0) 

20 or more 


What service are you in? 

68% (5) U.S. Navy 

8% (4) U.S. Army 

0% (3) U.S. Air Force 

8% (2) U.S. Marine Corps 

4% (1) U.S. Coast Guard 

12% (0) Foreign military or civilian 


7. How important do you consider classroom layout and seat¬ 
ing arrangement to effective learning? 

10% (5) Extremely important 

30% (4) Very important 

32% (3) Reasonably important 

12% (2) Mildly important 

12% (1) Not very important 

4% (0) Not a consideration 


8. What is your understanding of the primary purpose and 
use of the SOF? 

3% (5) For the superintendent to evaluate the 

professor 

35% (4) To give the department chairman an idea of 

the professor's performance or popularity 
33% (3) As feedback to the professor, only, to help 

improve teaching effectiveness 
10% (2) For pay, promotion, and tenure decisions 

6% (1) To influence the decision to keep, change, 

or delete the course 
13% (0) I have no idea. 
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9. 


What 


21% 

(5) 

50% 

(4) 

3% 

C3) 

9% 

(2) 

14% 

(1) 

3% 

(0) 


do you think the SOF should be used for? 

As a general feedback instrument for the pro¬ 
fessor's use only 

For use by the department chairman to evaluate 
the professor, in conjunction with other 
performance measures 

To rank a professor among other professors 
within the department 

For pay, promotion, and tenure decisions 
To influence the decision to keep, change, 
or delete the course 
SOF’s should not be used at all. 


10. How much weight do you think the SOF's should carry 
in pay, promotion, and tenure decisions? 

13% (5) 0% 

34% (4) 1-20% 

26% (3) 21-40% 

16% (2) 41-60% 

8% (1) 61-80% 

3% (0) 81-100% 


11. What is the primary source of your knowledge concerning 
the use of the SOF? 

4% (5) Curricular officer 

7% (4) Department chairman/academic associate 

34% (3) Individual professor 

10% (2) Welcome aboard/NPS indoctrination meetings 

24% (1) Other students 

21% (0) None 


12 . 


Which of the following evaluation methods do you feel 

would most increase teaching effectiveness? 

14% (5) Individual student conferences 

with the professor 

17% (4) Section leader conference with department 

chairman 

26% (3) Written narrative evaluation of professor 

by each student 

24% (2) Class visitation by department chairman 

or other faculty 
5% (1) Video-taping of class 

14% CO) Current SOF 


13. Do you think the professor sees the SOF's before 
grades are determined? 

3% (5) Yes, always 

9% (4) Frequently 

7% (3) If he/she requests to see them 

3% (2) If he/she has the permission of the 

department chairman 
42% (1) No, never. 

36% (0) I have no idea. 
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14. 


What percentage of your time at NPS have you completed 


SOF’s 

after you have known your course grade? 

55% 

137^% 

26% 

(4) 1-20% 

6% 

(3) 21-40% 

3% 

(2) 41-60% 

2% 

(1) 61-80% 

8% 

(0) 81-100% 


15. What percentage of your time at NPS have you completed 
SOF's outside of class? 

58% 137 '0% 

28% (4) 1-20% 

2% (3) 21-40% 

2% (2) 41-60% 

3% (1) 61-80% 

7% (0) 81-100% 

16. What percentage of your time at NPS have you completed 
SOF's in the presence of the professor? 

35% (5) 0% 

32% C4) 1-20% 

10% (3) 41-60% 

8% (2) 41-60% 

7% (1) 61-80% 

8% (0) 81-100% 

17. What percentage of your time at NPS have you felt that 

you have not been given enough time to complete the SOF? 
51% (5) 0% 

24% (4) 1-20% 

6% (3) 21-40% 

8% (2) 41-60% 

3% (1) 61-80% 

8% CO) 81-100% 

18. What do you think the average score for all professors 
in the Administrative Sciences Department is on SOF 
Question 12, "Overall, I would rate this instructor. . ."? 


6% 

C5) 

4.6-5.0 

38% 

(4) 

4.1-4.5 

38% 

(3) 

3.6-4.0 

15% 

C2) 

3.1-3.5 

3% 

CD 

2.6-3.0 

0% 

CO) 

2.5 or below 

Who 

do you 

. think sees the comments on the back of the 

SOF 

' s? 


32% 

C5) 

Only the professor 


3% (4) Only the department chairman 

4% C3) The department chairman and the academic 

associate 
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43% (2) The professor and the department chairman 

1% (1) The professor and the superintendent 

17% (0) The professor, academic associate, department 

chairman, and superintendent 

20. If you knew that only the professor saw the comments 

on the back of the SOF's, would you answer the questions 
on the front differently (either higher or lower)? 

2% (5) Yes, a lot higher 

4% (4) Yes, a little higher 

80% (3) No, I'd answer them about the same. 

5% (2) Yes, a little lower 

3% (1) Yes, a lot lower 

6% (0) I don't know. 

SAMPLE SIZE = 258 
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APPENDIX F 


FREQUENCIES OF SOF RESPONSES 


Ql--The course was well organized. 


Code 

Freq 

Percent 

0 

1 

0.1 

1 

16 

2.1 

2 

51 

6.6 

3 

94 

12.2 

4 

353 

45.9 

5 

252 

32.8 

Missing Values 

2 

0.3 

Total 

769 

100.0 

Mean: 4.005 

Mode: 4.000 



Q2--Time in class was 

spent effectively. 


Code 

Freq 

Percent 

0 

0 

0.0 

1 

21 

2.7 

2 

61 

7.9 

3 

123 

16.0 

4 

319 

41.5 

5 

244 

31.7 

Missing Values 

1 

0.1 

Total 

769 

100.0 

Mean: 3.917 

Mode: 4.000 



Q3--The instructor 

seemed to 

know when students 

didn' t 

understand the 

material. 



Code 


Freq 

Percent 

0 


4 

0.5 

1 


20 

2.6 

2 


38 

4.9 

3 


131 

17.0 

4 


322 

41.9 

5 


253 

32.9 

Missing Values 


1 

0.1 

Total 


769 

100.0 


Mean: 3.961 


Mode; 4.000 
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Q4--Di££icult concepts were made understandable 


Code 

Freq 

Percent 

0 

7 

0.9 

1 

22 

2.9 

2 

41 

5.3 

3 

145 

18.9 

4 

314 

40.8 

5 

239 

31.1 

Missing Values 

1 

0.1 

Total 

769 

100.0 

Mean: 3.893 

Mode: 4.000 



Q5--I had con£idence 
the subject. 

in the instructor 

’s knowledge o£ 

Code 

Freq 

Percent 

0 

1 

0.1 

1 

9 

1.2 

2 

17 

2.2 

3 

59 

7.7 

4 

227 

29.5 

5 

455 

59.2 

Missing Values 

1 

0.1 

Total 

769 

100.0 

Mean: 4.431 

Mode: 5. 

000 


Q6--I £elt £ree to 

ask questions. 


Code 

Freq 

Percent 

0 

0 

0.0 

1 

8 

1.0 

2 

17 

2.2 

3 

44 

5.7 

4 

242 

31.5 

5 

456 

59.3 

Missing Values 

2 

0.3 

Total 

769 

100.0 

Mean: 4.462 

Mode: 5.000 
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Q7--The instructor was prepared for the class. 


Code 

Freq 

Percent 

0 

6 

0.8 

1 

5 

0.7 

2 

13 

1.7 

3 

75 

9.8 

4 

298 

37.6 

5 

378 

49.2 

Missing Values 

3 

0.4 

Total 

769 

100.0 

Mean: 4.311 

Mode: 5.000 



Q8--The instructor' 

's objectives for 

the course 

have been 

made clear. 




Code 

Freq 


Percent 

0 

2 


0.3 

1 

20 


2.6 

2 

34 


4.4 

3 

126 


16.4 

4 

299 


38.9 

5 

287 


37.3 

Missing Values 

1 


0.1 

Total 

769 


100.0 

Mean: 4.033 

Mode: 

4.000 



Q9--The instructor made 
experience. 

this course 

a worthwhile learning 

Code 

Freq 

Percent 

0 

5 

0.7 

1 

27 

3.5 

2 

42 

5.5 

3 

117 

15.2 

4 

262 

34.1 

5 

315 

41.0 

Missing Values 

1 

0.1 


Total 


769 


100.0 


Mean; 4.017 


Mode: 5.000 
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Q10--The instructor stimulated my interest in the subject 
area. 


Code 

Freq 

Percent 

0 

5 

0.7 

1 

30 

3.9 

2 

52 

6.8 

3 

139 

18.1 

4 

258 

33.6 

5 

284 

36.9 

Missing Values 

1 

0.1 

Total 

769 

100.0 

Mean: 3.910 

Mode: 5.000 



Qll--The instructor cared 

about student progress 

and did 

his share in helping 

us to learn. 


Code 

Freq 

Percent 

0 

3 

0.4 

1 

13 

1.7 

2 

31 

4.0 

3 

119 

15.5 

4 

297 

38.6 

5 

304 

39.5 

Missing Values 

2 

0.3 

Total 

769 

100.0 

Mean; 4.094 

Mode: 5.000 


Q12--0verall, I would rate 

this instructor 


Code 

Freq 

Percent 

0 

2 

0.3 

1 

23 

3.0 

2 

40 

5.2 

3 

145 

18.9 

4 

269 

35.0 

5 

288 

37.5 

Missing Values 

2 

0.3 

Total 

769 

100.0 


Mean; 3.982 


Mode: 5.000 
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Q13--Overall, I would rate this course 


Code 

Freq 

Percent 

0 

1 

0.1 

1 

38 

4.9 

2 

60 

7.8 

3 

226 

29.4 

4 

290 

37.7 

5 

152 

19.8 

Missing Values 

2 

0.3 

Total 

769 

100.0 

Mean: 3.593 

Mode: 4.000 



Q14--Overall, I would rate 

the textbook(s) 


Code 

Freq 

Percent 

0 

43 

5.6 

1 

57 

7.4 

2 

74 

9.6 

3 

220 

28.6 

4 

232 

30.2 

5 

139 

18.1 

Missing Values 

4 

0.5 

Total 

769 

100.0 

Mean: 3.252 

Mode: 4.000 



Q15--Overall, I would rate 

the quality of the 

exams 

Code 

Freq 

Percent 

0 

83 

10.8 

1 

32 

4.2 

2 

50 

6.5 

3 

209 

27.2 

4 

263 

34.3 

5 

125 

16.3 

Missing Values 

7 

0.9 


Total 769 100.0 


Mean: 3.197 Mode: 4.000 


95 





I 


Q16--Overall, I would rate the laboratories 


Code 

Freq 

Percent 

0 

519 

67.5 

1 

21 

2.7 

2 

20 

2.6 

3 

50 

6.5 

4 

43 

5.6 

5 

57 

7.4 

Missing Values 

59 

7.7 

Total 

769 

100.0 

Mean: 0.941 

Mode: 0.000 


Q17--What is your 

anticipated grade for this course? 

Code 

Freq 

Percent 

0 

8 

1.0 

1 

20 

2.6 

2 

60 

7.8 

3 

161 

20.9 

4 

198 

25.7 

5 

114 

14.8 

Missing Values 

208 

27.0 

Total 

769 

100.0 

Mean: 3.538 

Mode: 4.000 


Q18--What is your 

total cumulative grade point 

average 

at the Naval 

Postgraduate School? 


Code 

Freq 

Percent 

0 

13 

1.7 

1 

20 

2.6 

2 

72 

9.4 

3 

146 

19.0 

4 

180 

23.4 

5 

108 

14.0 

Missing Values 

230 

29.0 

Total 

769 

100.0 


Mean: 3.455 


Mode: 4.000 














































Q19--Prior to this quarter, what were your expectations 

of this course in regard to interest, challenge, and 
potential usefulness in your career? 


Code 

Freq 

Percent 

0 

33 

4.3 

1 

19 

2.5 

2 

31 

4.0 

3 

156 

20.3 

4 

200 

26.0 

5 

122 

15.9 

Missing Values 

208 

27.0 

Total 

769 

100.0 

Mean: 3.492 

Mode: 4.000 


Q20--TO what extent 

do you think SOF's are 

worth your 

time and effort 

to fill out? 


Code 

Freq 

Percent 

0 

38 

4.9 

1 

52 

6.8 

2 

58 

7.5 

3 

236 

30.7 

4 

127 

16.5 

5 

49 

6.4 

Missing Values 

209 

27.2 


Total 


769 


100.0 


Mean: 2.909 


Mode: 3.000 
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